{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 一、神经网络简介\n",
    "\n",
    "感知机中，确定权重的工作（即：确定合适的、能符合预期的输入与输出的权重，还是由人工进行的）。而神经网络的一个重要性质就是：能够自动地从数据中学习到合适的权重参数。\n",
    "\n",
    "神经网络的示意图如下：\n",
    "\n",
    "![image-20230310192938882](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/image-20230310192938882.png)\n",
    "\n",
    "在该神经网络中，从左到右每一层依次编号0,1,2（从0开始编号，方便代码实现）。\n",
    "\n",
    "为了简化感知机的公式表示，现在引入一个函数$h(x)$，其中$x=b+x_1w_1+x_2w_2$，所以感知机公式可以简化如下：\n",
    "\n",
    "![image-20230310194123268](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/image-20230310194123268.png)\n",
    "\n",
    "同时，如果把$b$也看成一个输入，那么感知机的示意图就变成如下：\n",
    "\n",
    "![image-20230310214204807](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/image-20230310214204807.png)\n",
    "\n",
    "所以，一个感知机的输入信号的总和就可以写成$a=b·1+x_1w_1+x_2w_2$，该总和再经过一个过程处理后，送入到输出端，而这个处理过程就是一个函数，被称为**激活函数**，感知机的激活函数就是$h(x)$。\n",
    "\n",
    "整个过程表示如下：\n",
    "\n",
    "![image-20230310214744957](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/image-20230310214744957.png)\n",
    "\n",
    "`注：一般而言，“朴素感知机”是指单层网络，指的是激活函数使用了阶跃函数（超过一定阈值就切换输出的函数）的模型。“多层感知机”是指神经网络，即使用 sigmoid函数（后述）等平滑的激活函数的多层网络。`"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 二、激活函数\n",
    "\n",
    "感知机中使用了阶跃函数作为激活函数。如果将激活函数从阶跃函数换成其他函数，就可以进入神经网络的世界了。下面我们就来介绍一下神经网络使用的激活函数。\n",
    "\n",
    "## 2.1Sigmoid激活函数\n",
    "\n",
    "Sigmoid激活函数公式如下：\n",
    "\n",
    "![image-20230311212957752](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/image-20230311212957752.png)\n",
    "\n",
    "Sigmoid激活函数实现："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "\n",
    "def sigmoid_function(x):\n",
    "    y = 1/(1+np.exp(-x))\n",
    "    return y\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "用程序画出Sigmoid函数的图像："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAGdCAYAAADAAnMpAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAAA71klEQVR4nO3deXhU5eH28XtmkswkIQmBkASSkIRFFtkJRBYVaip1q/ZnlSoVRMVi0aq0bxUXqF2M1mpplYriWpVC1bpUEJeoUDSKBlBAFsOWEMhGQibrTDJz3j+CsZQtA0nOZOb7ua65kpyck7kzmpmbZ57zHIthGIYAAABMYjU7AAAACG6UEQAAYCrKCAAAMBVlBAAAmIoyAgAATEUZAQAApqKMAAAAU1FGAACAqULMDtAaXq9X+/fvV1RUlCwWi9lxAABAKxiGoerqavXq1UtW6/HHPzpFGdm/f79SUlLMjgEAAE5BYWGhkpOTj/v9TlFGoqKiJDX/MtHR0SanAQAAreF0OpWSktLyOn48naKMfPvWTHR0NGUEAIBO5mRTLJjACgAATEUZAQAApqKMAAAAU1FGAACAqSgjAADAVJQRAABgKsoIAAAwFWUEAACYyucysmbNGl1yySXq1auXLBaLXn/99ZMe89FHH2nUqFGy2+3q16+fnnvuuVOICgAAApHPZaS2tlbDhw/XokWLWrX/7t27ddFFF2ny5MnauHGjbrvtNt1www165513fA4LAAACj8/LwV9wwQW64IILWr3/4sWLlZ6erocffliSNGjQIK1du1Z//vOfNWXKFF/vHgAABJh2nzOSm5urrKysI7ZNmTJFubm57X3XAACgE2j3C+UVFxcrISHhiG0JCQlyOp2qr69XeHj4Uce4XC65XK6Wr51OZ3vHBACg02n0eFXf6FFDo0cNbq8ampo/dzV55Wr0ytV0+PMmj1yNXrk9XrmbvHI1edV4+HP34c9vPLevkroe/ZrcEfzyqr3Z2dm67777zI4BAECb8ngNVTc0ylnfJGdDo5wNjappaFJ1Q5NqXN/dag9/rHN5VOtuUp3bo1pXk+obPapze9Tg9qiu0SOP12izbJeOTArcMpKYmKiSkpIjtpWUlCg6OvqYoyKSNG/ePM2dO7fla6fTqZSUlHbNCQCALzxeQwdrXTpY41ZFrVvlNS5V1LpVWetWZV2jKurcOlTn1qG6RlXVN6qqrlHVrqZ2yWKxSI4QmxyhVjlCbXKE2mQPscr+7ccQq8JsVoUd/jz08OctN5tV8VH2dsnWGu1eRsaNG6eVK1cese29997TuHHjjnuM3W6X3W7egwIACF6GYaii1q0DVQ06UNWg4qp6lThdKnE2qKTapVJnQ0vxONWBifBQm6LDQxTlCFWU4/BHe4i62EMUaQ9RpN12+GOIIsNsiggLUUSYTRFhNoUf/jo81KbwUJvsoc0Fw2KxtO0D0YF8LiM1NTXKz89v+Xr37t3auHGjunXrpt69e2vevHkqKirS3//+d0nS7Nmz9dhjj+nXv/61rrvuOn3wwQf65z//qRUrVrTdbwEAgA+q6hq1t6JWew/WqaCiTvsq67Wvsk5Fh+pVVFkvV5O3VT/HYpG6RYSpe5cwdY+0q1uXMHWLCFNsRKi6RoQpNrL5Y9fwUMUcvkWHhyrUxpqj/83nMvLFF19o8uTJLV9/+3bKjBkz9Nxzz+nAgQMqKCho+X56erpWrFih22+/XX/5y1+UnJysp556itN6AQDtqsnjVUFFnXaU1GhnWY12l9dq1+GPlXWNJz2+R5RdPWMcSox2KDHGoYRoh+Kj7OoRZVd8lENxUc3FI4RicdoshmG03eyXduJ0OhUTE6OqqipFR0ebHQcA4GdKqxu09UC1th5wausBp7YXV2tXWa3cnuOPcMR1sSu1e4RSu0UopVuEkmLDlRwbruSuEUqMcSgshJJxulr7+u2XZ9MAAHA8B2tc2lh4SF/tq9LmoiptKqpSabXrmPuGh9rUL76L+sV3UZ+4SKX3iFR6XKTSukcq0s5LoL/gvwQAwG95vYbyy2q0bneF8vZWakNBpfYcrDtqP6tFSo+L1MCe0RrcM1oDE6N0RkKUkrqGy2rtvBM7gwVlBADgNwzDUH5pjdbmlyt350F9vqfimPM7+vaI1PCUrhqaFKOhSTEa3CtaEWG8pHVW/JcDAJjqUJ1bq3eUafWOMn2cX64S55FvuThCrRrVO1YZad00qndXjUyJVUxEqElp0R4oIwCADpdfWqN3vy7WB1tLtb6g8oj1OuwhVo1N76bxfeOU2aebhvSKYTJpgKOMAADanWEY+vqAU6s2F+vtzcXKL6054vsDEqI0aWAPndu/h0alxsoRajMpKcxAGQEAtJvd5bV6c+N+vfFlkXaV1bZsD7VZNL5vnLIGJ2jygB5Kjo0wMSXMRhkBALSp6oZG/fvLA/rnF4XaWHioZbs9xKrJA+J1wdBETR4Yr2gH8z7QjDICADhthmFofUGlXvqsQCs3HVBDY/NiY1aLNLF/D106vJfOPzNBURQQHANlBABwyhoaPXpz4349n7tHW/Y7W7b37RGpqWNSdNnIJMVHOUxMiM6AMgIA8FlZtUvPf7JHL362V4cOrwNiD7Hqh8N76Sdje2tU766d+iqy6FiUEQBAq+0qq9GS/+zWq+v3yX34yrbJseG65qxUXZmRotjIMJMTojOijAAATmpnWY3+mvON3vxyv769vOqIlK6afW4ffX9womwsuY7TQBkBABzX7vJa/TXnG72xsahlYbLvDYzX7HP7akxaLG/FoE1QRgAARymrdukvOTv0j3WF8hxuIVmDEnRbVn8NSYoxOR0CDWUEANCizt2kp/+zW4tX71St2yOpeSTk9qwzNDSZEoL2QRkBAMgwDK3cVKzfr/haB6oaJEnDkmN014WDdFaf7ianQ6CjjABAkMsvrdGCNzfr4/yDkqSkruH69Q8G6JJhvWRlYio6AGUEAIJUQ6NHj32QryfW7FSjx1BYiFU3ndtXN03qy4Xq0KEoIwAQhPL2VujXr3ylnYcvXpc1KF7zLz5TvbtzwTp0PMoIAASROneT/rhqu57P3SPDkHpE2fW7S8/UD4b0NDsaghhlBACCxKZ9Vbp12QbtKm8eDfnx6GTde9FgxURw8TqYizICAAHO4zX0xJqdeuTdHWryGkqMdujBHw/TuWf0MDsaIIkyAgABrcTZoFuXbdCnuyokSRcMSVT2/w1V1wiuIQP/QRkBgAD1yc5y/eIfG1Re41ZEmE2/+eGZumJ0Mku4w+9QRgAgwHi9hh5fvVMPv7tdXkMamBilv00bpT49upgdDTgmyggABJDqhkbdvvxLvb+1RFLzJNXfXTpE4WGsGwL/RRkBgABRcLBON/z9c+0oqVFYiFW/u/RMTR3T2+xYwElRRgAgAKzbXaHZL+apotathGi7lkzP0LDkrmbHAlqFMgIAndzLXxTqrtc2qdFjaFhyjJ68JkOJMQ6zYwGtRhkBgE7KMAwt+jBff3p3hyTpomE99acfD2d+CDodyggAdEIer6H7/r1Ff8/dK0m6aVJf/b/zB3CVXXRKlBEA6GQaGj26fflGvb25WBaLtODiwbp2QrrZsYBTRhkBgE6kzt2kWX//Qh/nH1SYzapHpg7XxcN6mR0LOC2UEQDoJGpcTbruuc+1bneFIsNsWjI9Q+P7xZkdCzhtlBEA6AScDY269pl1Wl9wSFH2ED133ViNTo01OxbQJigjAODnquoadc0zn+mrfVWKdoToheszNTylq9mxgDZDGQEAP1bjatL0Z9fpq31Vio0I1Ys3ZOrMXjFmxwLaFGUEAPxUvduj6577XF8WHlLXiFD948azNDAx2uxYQJuzmh0AAHA0V5NHP3sxT+t2VyjKHqIXrsukiCBgUUYAwM80eby6ZekGrdlRpvBQm56dOUZDk3lrBoGLMgIAfsQwDN3z+ma9+3WJwkKsempGhjLSupkdC2hXlBEA8CN/yflGyz4vlNUiPXbVSE1gHREEAcoIAPiJ5Z8XaOH730iSfnvpEJ1/ZqLJiYCOQRkBAD/wwbYS3fXaZknSzZP76adnpZqcCOg4lBEAMNnX+526eekGebyGLh+VrF+ef4bZkYAORRkBABOVVbt0w/Ofq87t0YR+3fXA5UNlsVjMjgV0KMoIAJjE1eTR7BfztL+qQelxkfrb1aMVauNpGcGH/+sBwASGYWjevzYpb2+loh0hempGhmIiQs2OBZiCMgIAJljyn1361/oi2awWLZo2Sn17dDE7EmAayggAdLBP8sv1wNvbJEn3XjRIZ/fvYXIiwFyUEQDoQAeq6nXLPzbIa0iXj0rWjPFpZkcCTEcZAYAO4m7y6ucvrdfBWrcG9YzW7y8bwpkzgCgjANBh/rDia20oOKQoR4gW/3SUwsNsZkcC/AJlBAA6wBsbi/R87l5J0sKpI5TaPdLkRID/oIwAQDvbe7BWdx9e6n3O5L46b1CCyYkA/0IZAYB25G7y6hf/2KAaV5PGpnXT7Vks9Q78r1MqI4sWLVJaWpocDocyMzO1bt26E+6/cOFCDRgwQOHh4UpJSdHtt9+uhoaGUwoMAJ3Jw+9u15f7qhQTHqqFPxmhEFZYBY7i81/F8uXLNXfuXC1YsEDr16/X8OHDNWXKFJWWlh5z/6VLl+rOO+/UggULtHXrVj399NNavny57rrrrtMODwD+bPWOMj2xZpck6cHLh6lX13CTEwH+yecy8sgjj2jWrFmaOXOmBg8erMWLFysiIkLPPPPMMff/5JNPNGHCBF199dVKS0vT+eefr6uuuuqkoykA0JmVVbv0y39ulCT99Kze+sGQRHMDAX7MpzLidruVl5enrKys736A1aqsrCzl5uYe85jx48crLy+vpXzs2rVLK1eu1IUXXnjc+3G5XHI6nUfcAKCzaL7uzFcqr3FrQEKU7rlosNmRAL8W4svO5eXl8ng8Skg4ciZ4QkKCtm3bdsxjrr76apWXl2vixIkyDENNTU2aPXv2Cd+myc7O1n333edLNADwGy/n7dP7W0sVZrNq4U9GyBHKeiLAibT7TKqPPvpI999/v/72t79p/fr1+te//qUVK1bod7/73XGPmTdvnqqqqlpuhYWF7R0TANpEYUWdfvvvryVJt3//DA3qGW1yIsD/+TQyEhcXJ5vNppKSkiO2l5SUKDHx2O+H3nvvvbrmmmt0ww03SJKGDh2q2tpa3Xjjjbr77rtltR7dh+x2u+x2uy/RAMB0Xq+h//fKl6pxNWl0aqxuPKeP2ZGATsGnkZGwsDCNHj1aOTk5Ldu8Xq9ycnI0bty4Yx5TV1d3VOGw2ZqHLA3D8DUvAPitZz/Zo093VSgizKZHrhwum5XrzgCt4dPIiCTNnTtXM2bMUEZGhsaOHauFCxeqtrZWM2fOlCRNnz5dSUlJys7OliRdcskleuSRRzRy5EhlZmYqPz9f9957ry655JKWUgIAnd2ushr9cVXz3Lm7LhzEcu+AD3wuI1OnTlVZWZnmz5+v4uJijRgxQqtWrWqZ1FpQUHDESMg999wji8Wie+65R0VFRerRo4cuueQS/eEPf2i73wIATOT1Grrz1U1yNXl1dv84TcvsbXYkoFOxGJ3gvRKn06mYmBhVVVUpOprJYAD8ywuf7tW9r29WRJhN795+jpJjI8yOBPiF1r5+sy4xAJyGokP1emDlVknSr6cMoIgAp4AyAgCnyDAM3fPaJtW6PRqdGqtrxqWZHQnolCgjAHCK3vxyvz7cXqYwm1UPXj6Us2eAU0QZAYBTUFnr1n2HFzf7xXn91C8+yuREQOdFGQGAU/Dgqm2qqG2+9szPzu1rdhygU6OMAICP8vZWaNnnzZep+MOPhijUxlMpcDr4CwIAHzR6vLr7tc2SpCszkpWR1s3kREDnRxkBAB889/EebSuuVmxEqO68YJDZcYCAQBkBgFbaf6hef35/hyRp3gWD1C0yzOREQGCgjABAK/3ura9V5/YoIzVWPx6dbHYcIGBQRgCgFT7OL9fbm4tls1r0+x8NkZU1RYA2QxkBgJNo8nh137+3SJJ+mtlbAxO5RhbQligjAHASL366VztKahQbEarbv3+G2XGAgEMZAYATOFjj0iPvNU9a/eX5A9Q1gkmrQFujjADACTz83g45G5o0qGe0rhrb2+w4QECijADAcWwuqtI/1hVIku774ZlcCA9oJ5QRADgGwzD0+xVfyzCki4f11Nh0VloF2gtlBACO4f2tpfp0V4XCQqy684KBZscBAhplBAD+R6PHq+yVWyVJ109MV3JshMmJgMBGGQGA/7H0swLtKq9V98gw/XxSX7PjAAGPMgIA/6WqvlELD19/5rbvn6EoR6jJiYDARxkBgP/ytw/zVVnXqH7xXXTVmBSz4wBBgTICAIcVVtTp2Y/3SJLuunCgQmw8RQIdgb80ADjskfd2yO3xanzf7po8IN7sOEDQoIwAgKStB5x6fWORJGneBYNksbDAGdBRKCMAIOmhd7bLMKSLhvXU0OQYs+MAQYUyAiDordtdoQ+2lSrEatGvzh9gdhwg6FBGAAQ1wzD0wNvNC5xNHZOi9LhIkxMBwYcyAiCovfd1idYXHJIj1KpfnNff7DhAUKKMAAhaHq+hh97ZLkm6bkK6EqIdJicCghNlBEDQemNjkb4prVFMeKh+di7LvgNmoYwACEqNHq8Wvv+NJGn2uX0VE86y74BZKCMAgtIreftUUFGnuC5hmjE+1ew4QFCjjAAIOq4mjx7NaR4VuWlSP0WEhZicCAhulBEAQWfZukLtr2pQYrRD0zJ7mx0HCHqUEQBBpd7t0WMf5kuSbv5ePzlCbSYnAkAZARBUXvh0j8qqXUqODdeVGSlmxwEgygiAIFLratLi1bskSbee119hITwFAv6Av0QAQeOFT/eqotattO4R+tHIJLPjADiMMgIgKNS5m/TkmuZRkZu/118hNp7+AH/BXyOAoPBCbvOoSGr3CF02opfZcQD8F8oIgIB3xKjI5H6MigB+hr9IAAHvxU/36uDhURHmigD+hzICIKDVuz0toyJzGBUB/BJ/lQAC2kuf7VV5jVu9uzEqAvgrygiAgNXQ6NET/zVXJJRREcAv8ZcJIGD984tClVW7lNQ1XD8axagI4K8oIwACkrvJq8Uf7ZQkzZ7Ul1ERwI/x1wkgIL22YZ/2VzUoPsquK0Ynmx0HwAlQRgAEnCaPV387PCpy4zl9uDIv4OcoIwACzopNB7T3YJ1iI0J1dWZvs+MAOAnKCICA4vUaeuyDfEnSDWf3UURYiMmJAJwMZQRAQHn362J9U1qjKEeIrhmXanYcAK1AGQEQMAzD0KIPm+eKzBiXpmhHqMmJALQGZQRAwFibX65NRVVyhFo1c0Ka2XEAtBJlBEDAePzwGTQ/GdNb3bvYTU4DoLVOqYwsWrRIaWlpcjgcyszM1Lp16064/6FDhzRnzhz17NlTdrtdZ5xxhlauXHlKgQHgWDYWHtInOw8qxGrRrHP6mB0HgA98nma+fPlyzZ07V4sXL1ZmZqYWLlyoKVOmaPv27YqPjz9qf7fbre9///uKj4/XK6+8oqSkJO3du1ddu3Zti/wAIEn624fNZ9BcOiJJSV3DTU4DwBc+l5FHHnlEs2bN0syZMyVJixcv1ooVK/TMM8/ozjvvPGr/Z555RhUVFfrkk08UGto8mSwtLe30UgPAf/mmpFrvfl0iSbppEqMiQGfj09s0brdbeXl5ysrK+u4HWK3KyspSbm7uMY958803NW7cOM2ZM0cJCQkaMmSI7r//fnk8ntNLDgCHPb66ea7I+YMT1C8+yuQ0AHzl08hIeXm5PB6PEhISjtiekJCgbdu2HfOYXbt26YMPPtC0adO0cuVK5efn6+c//7kaGxu1YMGCYx7jcrnkcrlavnY6nb7EBBBE9lXW6c2N+yVJP5/cz+Q0AE5Fu59N4/V6FR8fryeffFKjR4/W1KlTdffdd2vx4sXHPSY7O1sxMTEtt5SUlPaOCaCTenrtbjV5DY3r010jUrqaHQfAKfCpjMTFxclms6mkpOSI7SUlJUpMTDzmMT179tQZZ5whm+27C1UNGjRIxcXFcrvdxzxm3rx5qqqqarkVFhb6EhNAkKisdWvZuubnh5sm9TU5DYBT5VMZCQsL0+jRo5WTk9Oyzev1KicnR+PGjTvmMRMmTFB+fr68Xm/Lth07dqhnz54KCws75jF2u13R0dFH3ADgf73w6V7VN3o0uGe0zu4fZ3YcAKfI57dp5s6dqyVLluj555/X1q1bddNNN6m2trbl7Jrp06dr3rx5LfvfdNNNqqio0K233qodO3ZoxYoVuv/++zVnzpy2+y0ABJ2GRo+e+2SPJOln5/aRxWIxNxCAU+bzqb1Tp05VWVmZ5s+fr+LiYo0YMUKrVq1qmdRaUFAgq/W7jpOSkqJ33nlHt99+u4YNG6akpCTdeuutuuOOO9rutwAQdF7+olAVtW4lx4broqE9zY4D4DRYDMMwzA5xMk6nUzExMaqqquItGwBq8nj1vYdXq6CiTvf98EzNGJ9mdiQAx9Da12+uTQOg01m1pVgFFXWKjQjVFRnJZscBcJooIwA6FcMw9MTqXZKkGePTFBHm87vNAPwMZQRAp5K786A2FVXJEWrV9HFpZscB0AYoIwA6lSfWNI+KXJmRom6Rx14eAEDnQhkB0GlsK3Zq9Y4yWS3SDRO5IB4QKCgjADqNJWt2S5IuGNJTvbtHmJwGQFuhjADoFIqrGvTml0WSpFnnMCoCBBLKCIBO4dmPd6vRY2hsejcuiAcEGMoIAL9X3dCopZ8VSJJ+xqgIEHAoIwD83rJ1hap2Nalvj0hNHhBvdhwAbYwyAsCvNXq8eubj5omrN57TR1YrF8QDAg1lBIBfW/HVAR2oalBcF7suHZFkdhwA7YAyAsBvGYahJw8vcnbt+FQ5Qm0mJwLQHigjAPxW7s6D+vqAU+GhNk3LTDU7DoB2QhkB4Lee/E/zqMgVGcmKZel3IGBRRgD4pR0l1fpoe5ksFun6ielmxwHQjigjAPzSU4dHRaYMTlRq90iT0wBoT5QRAH6ntLpBr2/YL4ml34FgQBkB4Hf+/sleuT1ejerdVaNTY82OA6CdUUYA+JU6d5Ne/GyvJGnW2YyKAMGAMgLAr7yat0+H6hqV2j1C55+ZaHYcAB2AMgLAb3i8hp5e27z0+3UT0mVj6XcgKFBGAPiN97eWaM/BOsWEh+qKjGSz4wDoIJQRAH7j29N5p2X2VkRYiMlpAHQUyggAv7Cx8JA+31OpUJtFM8anmR0HQAeijADwC0sOj4r8cHiSEqIdJqcB0JEoIwBMV1hRp7c3HZAk3XA2S78DwYYyAsB0z368R15DOrt/nAb1jDY7DoAORhkBYKqq+kYt/7xAknQDi5wBQYkyAsBUy9YVqNbt0YCEKJ3TP87sOABMQBkBYJpGj1fPfbJHknT92emyWFjkDAhGlBEAplnx1QEdqGpQjyi7Lh3Ry+w4AExCGQFgCsMwWk7nnTEuVfYQm8mJAJiFMgLAFLm7DmrLfqccoVZNy0w1Ow4AE1FGAJjiqf80XxDvx6OTFRsZZnIaAGaijADocPmlNfpgW6ksFun6iZzOCwQ7ygiADvf02ua5IlmDEpQeF2lyGgBmo4wA6FBl1S69ur5IknTjOYyKAKCMAOhgL3y6V+4mr0akdFVGaqzZcQD4AcoIgA5T7/bohdw9kqRZZ/dhkTMAkigjADrQq+v3qbKuUSndwjXlzASz4wDwE5QRAB3C6zX09Nrm03mvm5CuEBtPPwCa8WwAoEO8v7VEu8trFe0I0ZUZKWbHAeBHKCMAOsS3S79POytVkfYQk9MA8CeUEQDtbn1BpT7fU6lQm0XXjk8zOw4AP0MZAdDulqxpHhW5dESSEqIdJqcB4G8oIwDa1Z7yWq3aUiyJRc4AHBtlBEC7enrtbhmGNHlAD52REGV2HAB+iDICoN1U1Lr1cl6hJGkWoyIAjoMyAqDdvJC7Vw2NXg1NitG4Pt3NjgPAT1FGALSLhkaP/n546fcbz2HpdwDHRxkB0C5eydung7VuJceG64IhiWbHAeDHKCMA2pzHa+ipw4ucXT+Rpd8BnBjPEADa3DtbirXnYJ26RoSy9DuAk6KMAGhThmHoidU7JUnTx6Wx9DuAk6KMAGhTubsO6st9VXKEWjVjXKrZcQB0ApQRAG3qidXNc0WuzEhR9y52k9MA6AxOqYwsWrRIaWlpcjgcyszM1Lp161p13LJly2SxWHTZZZedyt0C8HNf73dq9Y4yWS3SDRNZ5AxA6/hcRpYvX665c+dqwYIFWr9+vYYPH64pU6aotLT0hMft2bNHv/rVr3T22WefclgA/u3JNc1zRS4a1ku9u0eYnAZAZ+FzGXnkkUc0a9YszZw5U4MHD9bixYsVERGhZ5555rjHeDweTZs2Tffdd5/69OFfS0AgKqyo07+/OiBJ+hlLvwPwgU9lxO12Ky8vT1lZWd/9AKtVWVlZys3NPe5xv/3tbxUfH6/rr7++VffjcrnkdDqPuAHwb0+v3S2P19DEfnEakhRjdhwAnYhPZaS8vFwej0cJCQlHbE9ISFBxcfExj1m7dq2efvppLVmypNX3k52drZiYmJZbSgrrFAD+7GCNS8s+L5AkzT63r8lpAHQ27Xo2TXV1ta655hotWbJEcXFxrT5u3rx5qqqqarkVFha2Y0oAp+vZj/eoodGrYckxmtCPC+IB8I1PqxHFxcXJZrOppKTkiO0lJSVKTDz62hM7d+7Unj17dMkll7Rs83q9zXccEqLt27erb9+j/xVlt9tlt3NKINAZVDc0tlwQ7+eT+nJBPAA+82lkJCwsTKNHj1ZOTk7LNq/Xq5ycHI0bN+6o/QcOHKhNmzZp48aNLbcf/vCHmjx5sjZu3MjbL0AAWPpZgZwNTerbI1LnD+aCeAB85/M6zXPnztWMGTOUkZGhsWPHauHChaqtrdXMmTMlSdOnT1dSUpKys7PlcDg0ZMiQI47v2rWrJB21HUDn09Do0VNrd0tqnititTIqAsB3PpeRqVOnqqysTPPnz1dxcbFGjBihVatWtUxqLSgokNXKwq5AMHh1/T6VVbvUM8ahS0ckmR0HQCdlMQzDMDvEyTidTsXExKiqqkrR0dFmxwEgqcnj1fceXq2CijrNv3iwrpuYbnYkAH6mta/fDGEAOCUrNh1QQUWdYiNC9ZOxzP8CcOooIwB85vUa+tuHzUu/z5yQrogwn9/xBYAWlBEAPntva4m2l1Qryh6iGePTzI4DoJOjjADwiWEYeuyDfEnS9PGpigkPNTkRgM6OMgLAJ6t3lGlTUZXCQ226bgKTVgGcPsoIgFYzDEOPHh4VmZbZW927sFIygNNHGQHQap/uqlDe3kqFhVg165w+ZscBECAoIwBabdGHzaMiUzNSlBDtMDkNgEBBGQHQKnl7K7U2v1whVot+di6jIgDaDmUEQKv8JecbSdL/jUpScmyEyWkABBLKCICTWl9QqTU7ymSzWnTz5P5mxwEQYCgjAE7qL+8fHhUZmaTe3RkVAdC2KCMATmhDQaVWfzsq8r1+ZscBEIAoIwBO6Nu5Ij8amaTU7pEmpwEQiCgjAI5rY+EhfbT927kijIoAaB+UEQDH9dfDoyKXjUhSWhyjIgDaB2UEwDGtL6jUB9tKmSsCoN1RRgAc0yPv7pDUfAZNOqMiANoRZQTAUT7ddVBr88sVarPoF+exrgiA9kUZAXAEwzBaRkWmjklRSjfWFQHQvigjAI7wn2/KtW5PhcJCrKy2CqBDUEYAtDAMQw+/u12SdM1ZqUqM4cq8ANofZQRAi5ytpfpyX5XCQ226aVJfs+MACBKUEQCSJK/X0J8Oj4rMnJCmuC52kxMBCBaUEQCSpDe+LNK24mpFOUJ04zl9zI4DIIhQRgDI1eTRw4fPoLlpUl91jQgzORGAYEIZAaClnxVoX2W9EqLtmjk+3ew4AIIMZQQIcjWuJj32Qb4k6dbzzlB4mM3kRACCDWUECHJL1uzSwVq3+sRF6sqMZLPjAAhClBEgiJXXuPTUf3ZJkn41ZYBCbDwlAOh4PPMAQeyvOd+o1u3RsOQYXTAk0ew4AIIUZQQIUvmlNXrpswJJ0p0XDJTFYjE5EYBgRRkBgtQDb2+Vx2soa1C8xveNMzsOgCBGGQGC0Cc7y/X+1lLZrBbdecEgs+MACHKUESDIeL2G/rBiqyRpWmZv9YvvYnIiAMGOMgIEmdc2FGnLfqei7CG69bz+ZscBAMoIEEzq3R499E7zxfDmfK+funMxPAB+gDICBJEn1uxUsbNBSV3Dde34NLPjAIAkyggQNPZV1unxj3ZKku66cJAcoSz7DsA/UEaAIJG9cptcTV6d1aebLhzKAmcA/AdlBAgCn+ws14pNB2S1SAsuOZMFzgD4FcoIEOCaPF7d9+bXkqSfnpWqQT2jTU4EAEeijAABbum6Am0vqVbXiFDN/f4ZZscBgKNQRoAAdrDGpYff3SFJ+uX5A9Q1IszkRABwNMoIEMCy396mqvpGDeoZravH9jY7DgAcE2UECFDrdlfolbx9slikP/xoiGxWJq0C8E+UESAAuZu8uuf1TZKkn4zprVG9Y01OBADHRxkBAtDTa3drR0mNukeG6Y4fDDA7DgCcEGUECDD7Kuv015xvJEnzLhzEpFUAfo8yAgQQwzD0mze3qL7Ro7Hp3XT5qCSzIwHASVFGgACyYtMBvb+1VKE2i35/2RBWWgXQKVBGgABRWevWgje2SJJ+PqmfzkiIMjkRALQOZQQIEL9762sdrHXrjIQu+vnkvmbHAYBWo4wAAeDD7aX614YiWSzSg5cPkz3EZnYkAGg1ygjQydW4mnT3v5rXFLluQrpGsqYIgE6GMgJ0cg++vU37qxqU0i1cvzyfC+EB6HwoI0AntmZHmV74dK8k6YH/G6aIsBCTEwGA706pjCxatEhpaWlyOBzKzMzUunXrjrvvkiVLdPbZZys2NlaxsbHKyso64f4AWqeqrlG/fuUrSdKMcama0C/O5EQAcGp8LiPLly/X3LlztWDBAq1fv17Dhw/XlClTVFpaesz9P/roI1111VX68MMPlZubq5SUFJ1//vkqKio67fBAMPvNv7eo2NmgPnGRuvOCQWbHAYBTZjEMw/DlgMzMTI0ZM0aPPfaYJMnr9SolJUW33HKL7rzzzpMe7/F4FBsbq8cee0zTp09v1X06nU7FxMSoqqpK0dHRvsQFAtLKTQf085fWy2qRXrlpPBfCA+CXWvv67dPIiNvtVl5enrKysr77AVarsrKylJub26qfUVdXp8bGRnXr1u24+7hcLjmdziNuAJqVVjfo7teaz575+aR+FBEAnZ5PZaS8vFwej0cJCQlHbE9ISFBxcXGrfsYdd9yhXr16HVFo/ld2drZiYmJabikpKb7EBAKW12voVy9/pcq6Rg3uGa1fnNff7EgAcNo69GyaBx54QMuWLdNrr70mh8Nx3P3mzZunqqqqllthYWEHpgT811Nrd2nNjjI5Qq1a+JMRCgvhhDgAnZ9P5wHGxcXJZrOppKTkiO0lJSVKTEw84bF/+tOf9MADD+j999/XsGHDTriv3W6X3W73JRoQ8L4sPKQ/rtouSZp/8ZlcewZAwPDpn1VhYWEaPXq0cnJyWrZ5vV7l5ORo3Lhxxz3uj3/8o373u99p1apVysjIOPW0QJCqbmjULf/YoCavoQuHJuqqsbx1CSBw+LxC0ty5czVjxgxlZGRo7NixWrhwoWprazVz5kxJ0vTp05WUlKTs7GxJ0oMPPqj58+dr6dKlSktLa5lb0qVLF3Xp0qUNfxUgMBmGoXte36yCijoldQ1X9o+GyWKxmB0LANqMz2Vk6tSpKisr0/z581VcXKwRI0Zo1apVLZNaCwoKZLV+N+Dy+OOPy+1268c//vERP2fBggX6zW9+c3rpgSCw/PNCvbFxv2xWi/561QjFRISaHQkA2pTP64yYgXVGEKw27avS5Ys/kbvJq/83ZYDmTO5ndiQAaLV2WWcEQMeprHVr9ot5cjd5lTUoXjed29fsSADQLigjgB/yeg3dtnyjig7VK7V7hB6+coSsVuaJAAhMlBHAD/31g2+0+vB6Io9PG62YcOaJAAhclBHAz7yzpVgL3/9GkvSHy4ZqcC/mSQEIbJQRwI98vd+p25dvlCTNGJeqy0cnmxsIADoAZQTwE2XVLs36+xeqc3s0sV+c7r14sNmRAKBDUEYAP+Bq8mj2i3kqOlSv9LhILbp6lEJs/HkCCA482wEmMwxD817dpLy9lYpyhOipGRksbAYgqFBGAJM9/O4O/WtDkWxWixZdPUp9e3CZBADBhTICmOiFT/fqsQ/zJUl/uGyIzjmjh8mJAKDjUUYAk7y7pVgL3tgsSbr1vP76ydjeJicCAHNQRgAT5O2t0C3/2CCvIU3NSNFtWf3NjgQApqGMAB1sc1GVrn32c7mavJo8oIf+8KMhslhY6h1A8KKMAB1oR0m1rnn6M1U3NCkjNVaLpnEKLwDwLAh0kD3ltfrpU5+psq5Rw5Jj9MzMMYoICzE7FgCYjjICdIB9lXWa9tRnKq12aUBClJ6fOVbRDtYSAQCJMgK0u70HazX1iU9VdKhefeIi9eINmYqNDDM7FgD4DcaIgXa0s6xGVy/5VCVOl/rEReqlWZnqEWU3OxYA+BXKCNBOdpRU6+oln6m8xqX+8V300qxMxUc5zI4FAH6HMgK0g6/2HdK1z36uilq3BvWM1ovXj1X3LoyIAMCxUEaANrZmR5lmv5inOrdHQ5Ni9ML1Y9U1gjkiAHA8lBGgDb22YZ/+38tfqclraGK/OC2+ZrS62PkzA4AT4VkSaAOGYejJNbuU/fY2SdIPh/fSn64YrrAQTlgDgJOhjACnyd3k1b2vb9byLwolSddNSNc9Fw2S1coS7wDQGpQR4DRU1Lo1+8U8rdtdIatFuuvCQbp+YjrXmgEAH1BGgFO0o6RaNzz/hQoq6hRlD9Ffrx6pyQPizY4FAJ0OZQQ4BW9sLNK8f21Snduj3t0i9PSMDPVPiDI7FgB0SpQRwAeuJo/+sGKr/p67V5I0oV93PXrVKHVjeXcAOGWUEaCV9lXWac7SDfqy8JAk6ebJ/XT798+QjYmqAHBaKCNAK7yxsUj3vL5Z1Q1NigkP1Z+nDtf3BiaYHQsAAgJlBDiBqvpGzX9js97YuF+SNKp3V/3lJyOV0i3C5GQAEDgoI8BxrP2mXHe8+pWKDtXLZrXolu/1082T+ynExkJmANCWKCPA/6iqb9T9K7a2LGLWu1uE/jx1hEanxpqcDAACE2UE+C/vfV2ie17fpBKnS5I0fVyqfv2DgVxfBgDaEc+wgKTCijrd9+8ten9rqSQpPS5SD14+TGPTu5mcDAACH2UEQa2h0aPFq3fq8Y92ytXkVYjVohvO7qPbsvrLEWozOx4ABAXKCIKS12vo31/t10PvbNe+ynpJzQuY3ffDM9UvnpVUAaAjUUYQdD7ZWa7sldu0qahKktQzxqF7LhqsC4cmcoE7ADABZQRBY31BpRa+/43W7CiTJHWxh+imSX113YR0hYfxlgwAmIUygoC3sfCQ/vzeDq0+XEJCrBZdndlbvzivv+K62E1OBwCgjCAgGYahj/MPavHqnVqbXy5Jslkt+r+RSbrle/3VuzsrqAKAv6CMIKA0erxauemAnlyzS1v2OyU1l5DLRiTplu/1U1pcpMkJAQD/izKCgFDqbNA/1hXqpc/2qrS6ecGy8FCbpo5J0fUT07mWDAD4McoIOi2v19DHO8u1/PNCrdpcrCavIUmK62LXNWelavq4VMVGhpmcEgBwMpQRdDqFFXV6JW+fXsnbp6JD9S3bM1Jjdc24VF0wpKfCQriYHQB0FpQRdArlNS6t+OqA3thYpPUFh1q2RztCdNnIJF2ZkaIhSTHmBQQAnDLKCPxWibNB724p1qotxfp0V4U8h9+GsVqk8X3jdEVGsqacmciy7QDQyVFG4DcMw9DWA9X6cHup3t9aog3/NQIiScNTuurS4b108bCeio92mBMSANDmKCMwVUWtW7k7D2ptfpk+3FamYmfDEd8f1burfjAkUVPOTFRqd07LBYBARBlBh6qqa9Tneyq0bk+FPtlZri37nTKM777vCLVqYr84TR4Yr6xBCUpgBAQAAh5lBO3GMAztOVinDQWV2lBwSF/srdS24iPLhyQNSIjShH5xOueMOJ3VpztzQAAgyFBG0CYMw9D+qgZt2lelTUWHtKnIqU37DqmyrvGoffv0iFRmejdlpnfX+H7dFR/F6AcABDPKCHxWVd+o/NJqfVNSo23F1fr6gFPbDjjlbGg6at+wEKuGJsVoZEpXjUqN1Zi0buoRxcXpAADfoYzgmNxNXhUdqteeg7XaXVarXeU12lVWq51lNSpxuo55TIjVojMSojQ0KUZDkmM0NClGg3pGyR7C2y4AgOOjjAQpd5NXJc4GHahqUNGhOu2rqNe+ynrtO1SnvQfrtP9QvbzG8Y/vGeNQv/guGpAQpUE9ozWoZ7T6xkdSPAAAPqOMBJiGRo8qat0qq3apvMalsmqXSqtdKnE2qMTpUml1cwEpr3EdNZH0f4WH2tS7W4T69IhUnx6RSo/roj49ItU/vouiHKEd8wsBAALeKZWRRYsW6aGHHlJxcbGGDx+uRx99VGPHjj3u/i+//LLuvfde7dmzR/3799eDDz6oCy+88JRDB4NGj1fVDU1y1jfK2dCoqvrm26G65o+VtW5V1jWqss6tyjq3KmrdOljjVo3r6HkbxxMWYlXPGIeSuoYrOTZcSV0jlBwbrt7dI5TaLUI9ouyyWCzt+FsCAHAKZWT58uWaO3euFi9erMzMTC1cuFBTpkzR9u3bFR8ff9T+n3zyia666iplZ2fr4osv1tKlS3XZZZdp/fr1GjJkSJv8EmZq8njlavKqodGjhiav6t2e5s8bPapzN9+++7xJdW6Pat1NqnU1qdblaf7oblKNy6PqhkZVNzSppqFJ9Y2eU84UarMorotdcV3s6hFlV1yXMCVGOxQf7VBCtEOJ0Q717OpQ98gwygYAwHQWwzjZYP2RMjMzNWbMGD322GOSJK/Xq5SUFN1yyy268847j9p/6tSpqq2t1VtvvdWy7ayzztKIESO0ePHiVt2n0+lUTEyMqqqqFB0d7UvcE3p67W4VVtTJ7fGqscmrRo9Xbo9X7iaj+fOmb7/2ytXkOfyx+eY+XECaTjSxog1EhtkUHR6qaEeoukaEKia8+RYbGabYiDDFRoSqa0SY4rqEqVtkmLp3sSvaEULJAACYrrWv3z6NjLjdbuXl5WnevHkt26xWq7KyspSbm3vMY3JzczV37twjtk2ZMkWvv/76ce/H5XLJ5frujA2n0+lLzFZ766v9R13/5HSE2awKD7PJEWpVeKhNjlCbIsJsiggLkSPUpkh78+eRYc3buzhCFGkPURd7iCLDQhTlCFEXR4ii7KHq4ghRtCNEITZrm+UDAMAf+VRGysvL5fF4lJCQcMT2hIQEbdu27ZjHFBcXH3P/4uLi495Pdna27rvvPl+inZLLRyVrYr84hdqsh28WhdqsCguxKsxmVYjNInuITfYQq+whzdvtITbZQ7/72hHSXDrsIVZZrYxGAADgK788m2bevHlHjKY4nU6lpKS0+f389KzUNv+ZAADANz6Vkbi4ONlsNpWUlByxvaSkRImJicc8JjEx0af9Jclut8tuZ5VOAACCgU8TEsLCwjR69Gjl5OS0bPN6vcrJydG4ceOOecy4ceOO2F+S3nvvvePuDwAAgovPb9PMnTtXM2bMUEZGhsaOHauFCxeqtrZWM2fOlCRNnz5dSUlJys7OliTdeuutOvfcc/Xwww/roosu0rJly/TFF1/oySefbNvfBAAAdEo+l5GpU6eqrKxM8+fPV3FxsUaMGKFVq1a1TFItKCiQ1frdgMv48eO1dOlS3XPPPbrrrrvUv39/vf766wGxxggAADh9Pq8zYob2WmcEAAC0n9a+frOIBQAAMBVlBAAAmIoyAgAATEUZAQAApqKMAAAAU1FGAACAqSgjAADAVJQRAABgKr+8au//+nZdNqfTaXISAADQWt++bp9sfdVOUUaqq6slSSkpKSYnAQAAvqqurlZMTMxxv98ploP3er3av3+/oqKiZLFY2uznOp1OpaSkqLCwkGXmW4HHyzc8Xr7h8Wo9Hivf8Hj5pi0fL8MwVF1drV69eh1x3br/1SlGRqxWq5KTk9vt50dHR/M/qA94vHzD4+UbHq/W47HyDY+Xb9rq8TrRiMi3mMAKAABMRRkBAACmCuoyYrfbtWDBAtntdrOjdAo8Xr7h8fINj1fr8Vj5hsfLN2Y8Xp1iAisAAAhcQT0yAgAAzEcZAQAApqKMAAAAU1FGAACAqSgj/2XFihXKzMxUeHi4YmNjddlll5kdye+5XC6NGDFCFotFGzduNDuOX9qzZ4+uv/56paenKzw8XH379tWCBQvkdrvNjuY3Fi1apLS0NDkcDmVmZmrdunVmR/JL2dnZGjNmjKKiohQfH6/LLrtM27dvNztWp/DAAw/IYrHotttuMzuKXysqKtJPf/pTde/eXeHh4Ro6dKi++OKLdr9fyshhr776qq655hrNnDlTX375pT7++GNdffXVZsfye7/+9a/Vq1cvs2P4tW3btsnr9eqJJ57Qli1b9Oc//1mLFy/WXXfdZXY0v7B8+XLNnTtXCxYs0Pr16zV8+HBNmTJFpaWlZkfzO6tXr9acOXP06aef6r333lNjY6POP/981dbWmh3Nr33++ed64oknNGzYMLOj+LXKykpNmDBBoaGhevvtt/X111/r4YcfVmxsbPvfuQGjsbHRSEpKMp566imzo3QqK1euNAYOHGhs2bLFkGRs2LDB7Eidxh//+EcjPT3d7Bh+YezYscacOXNavvZ4PEavXr2M7OxsE1N1DqWlpYYkY/Xq1WZH8VvV1dVG//79jffee88499xzjVtvvdXsSH7rjjvuMCZOnGjKfTMyImn9+vUqKiqS1WrVyJEj1bNnT11wwQXavHmz2dH8VklJiWbNmqUXXnhBERERZsfpdKqqqtStWzezY5jO7XYrLy9PWVlZLdusVquysrKUm5trYrLOoaqqSpL4f+kE5syZo4suuuiI/8dwbG+++aYyMjJ0xRVXKD4+XiNHjtSSJUs65L4pI5J27dolSfrNb36je+65R2+99ZZiY2M1adIkVVRUmJzO/xiGoWuvvVazZ89WRkaG2XE6nfz8fD366KP62c9+ZnYU05WXl8vj8SghIeGI7QkJCSouLjYpVefg9Xp12223acKECRoyZIjZcfzSsmXLtH79emVnZ5sdpVPYtWuXHn/8cfXv31/vvPOObrrpJv3iF7/Q888/3+73HdBl5M4775TFYjnh7dv38yXp7rvv1uWXX67Ro0fr2WeflcVi0csvv2zyb9FxWvt4Pfroo6qurta8efPMjmyq1j5e/62oqEg/+MEPdMUVV2jWrFkmJUcgmDNnjjZv3qxly5aZHcUvFRYW6tZbb9VLL70kh8NhdpxOwev1atSoUbr//vs1cuRI3XjjjZo1a5YWL17c7vcd0u73YKJf/vKXuvbaa0+4T58+fXTgwAFJ0uDBg1u22+129enTRwUFBe0Z0a+09vH64IMPlJube9R1CzIyMjRt2rQOadH+oLWP17f279+vyZMna/z48XryySfbOV3nEBcXJ5vNppKSkiO2l5SUKDEx0aRU/u/mm2/WW2+9pTVr1ig5OdnsOH4pLy9PpaWlGjVqVMs2j8ejNWvW6LHHHpPL5ZLNZjMxof/p2bPnEa+DkjRo0CC9+uqr7X7fAV1GevTooR49epx0v9GjR8tut2v79u2aOHGiJKmxsVF79uxRampqe8f0G619vP7617/q97//fcvX+/fv15QpU7R8+XJlZma2Z0S/0trHS2oeEZk8eXLLqJvVGtCDkq0WFham0aNHKycnp+VUeq/Xq5ycHN18883mhvNDhmHolltu0WuvvaaPPvpI6enpZkfyW+edd542bdp0xLaZM2dq4MCBuuOOOygixzBhwoSjThXfsWNHh7wOBnQZaa3o6GjNnj1bCxYsUEpKilJTU/XQQw9Jkq644gqT0/mf3r17H/F1ly5dJEl9+/blX2nHUFRUpEmTJik1NVV/+tOfVFZW1vI9/vUvzZ07VzNmzFBGRobGjh2rhQsXqra2VjNnzjQ7mt+ZM2eOli5dqjfeeENRUVEt82piYmIUHh5ucjr/EhUVddRcmsjISHXv3p05Nsdx++23a/z48br//vt15ZVXat26dXryySc7ZCSXMnLYQw89pJCQEF1zzTWqr69XZmamPvjgg445vxoB7b333lN+fr7y8/OPKmsGF83W1KlTVVZWpvnz56u4uFgjRozQqlWrjprUCunxxx+XJE2aNOmI7c8+++xJ3zIETmbMmDF67bXXNG/ePP32t79Venq6Fi5cqGnTprX7fVsMng0BAICJeOMaAACYijICAABMRRkBAACmoowAAABTUUYAAICpKCMAAMBUlBEAAGAqyggAADAVZQQAAJiKMgIAAExFGQEAAKaijAAAAFP9f5c1zCUln3dJAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "x = np.arange(-6, 6, 0.1)\n",
    "y = sigmoid_function(x)\n",
    "\n",
    "plt.plot(x, y)\n",
    "plt.show()\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2阶跃激活函数\n",
    "\n",
    "阶跃激活函数公式如下：\n",
    "\n",
    "![image-20230310194123268](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/image-20230310194123268.png)\n",
    "\n",
    "阶跃激活函数的实现："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 输入为numpy数组的阶跃函数\n",
    "import numpy as np\n",
    "\n",
    "\n",
    "def step_function(x):  # x为numpy数组\n",
    "    y = x > 0  # y为bool型数组\n",
    "    return y.astype(int)  # 转换数据类型\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面用代码画出阶跃函数的图像："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAGdCAYAAADAAnMpAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAj0ElEQVR4nO3df3BU1f3/8dcmmA1UEtHIhh8pQWoLDJpAIpmAjjBNjdahw0xrGbWA+WosNrTAthWDmIhaYlUwrUYjKNbpyJCq1TpC49BYdKxpqYn0q1ZwKKXJB8xChjabRpvQ3f38AbshHwJmgeSc5DwfMztMLvfmvrPjmhfnnPe5nkgkEhEAAIAhCaYLAAAAbiOMAAAAowgjAADAKMIIAAAwijACAACMIowAAACjCCMAAMAowggAADBqmOkC+iIcDuvgwYMaOXKkPB6P6XIAAEAfRCIRtbe3a+zYsUpIOPX4x6AIIwcPHlRGRobpMgAAwBlobm7W+PHjT/n3gyKMjBw5UtKxHyYlJcVwNQAAoC+CwaAyMjJiv8dPZVCEkejUTEpKCmEEAIBB5vOWWLCAFQAAGEUYAQAARhFGAACAUYQRAABgFGEEAAAYRRgBAABGEUYAAIBRhBEAAGAUYQQAABgVdxh56623NG/ePI0dO1Yej0evvPLK516zY8cOzZgxQ16vV1/60pf0i1/84gxKBQAAQ1HcYaSjo0NZWVmqqqrq0/l///vfdf3112vu3LnatWuXli9frttuu02vv/563MUCAIChJ+5n01x33XW67rrr+nx+dXW1Jk6cqHXr1kmSpkyZorfffluPPvqoCgsL4709AAAYYvr9QXn19fUqKCjocaywsFDLly8/5TWdnZ3q7OyMfR0MBvurPAAWaj7yqZ7/U5M6/xsyXQrgjP83e6IyLhxh5N79HkZaWlrk8/l6HPP5fAoGg/rss880fPjwk66pqKjQmjVr+rs0AJZ68s2/afOfmkyXAThlXtbYoRtGzkRpaan8fn/s62AwqIyMDIMVARhI7f/5ryTpyi+lKSsj1XA1gBt8KcnG7t3vYSQ9PV2BQKDHsUAgoJSUlF5HRSTJ6/XK6/X2d2kALBUORyRJBVNG65bZEw1XA6C/9fs+I/n5+aqrq+txbPv27crPz+/vWwMYpELHw0higsdwJQAGQtxh5N///rd27dqlXbt2STrWurtr1y41NR2b3y0tLdWiRYti5y9ZskT79u3TnXfeqd27d+uJJ57Qr371K61YseLc/AQAhpxQ5FgYSSCMAE6IO4y8++67mj59uqZPny5J8vv9mj59usrKyiRJn3zySSyYSNLEiRO1detWbd++XVlZWVq3bp2efvpp2noBnFLkeBhJ9BBGABfEvWZkzpw5sf9R9Ka33VXnzJmj9957L95bAXBUdJomgTACOIFn0wCwTuj4v3eYpgHcQBgBYJ1wbAGr4UIADAg+6gCswzQN4BbCCADrRLtpaO0F3EAYAWCd2DQNIyOAEwgjAKzDPiOAWwgjAKzDyAjgFsIIAOsczyKsGQEcQRgBYJ1oNw0DI4AbCCMArBOmmwZwCmEEgHVCrBkBnEIYAWAdumkAtxBGAFinezt4wgjgAsIIAOvERkaYpgGcQBgBYJ1w+NifjIwAbiCMALAOC1gBtxBGAFgnHFvAargQAAOCjzoA64RZMwI4hTACwDohumkApxBGAFgnGkYYGQHcQBgBYB0elAe4hTACwDp00wBuIYwAsE6IbhrAKXzUAViH7eABtxBGAFgnOjLCNA3gBsIIAKtEIhEdzyLyEEYAJxBGAFgl2kkjMU0DuIIwAsAqoRPSCNM0gBsIIwCsEt0KXqKbBnAFH3UAVukxMsI0DeAEwggAq4ROHBlhmgZwAmEEgFXCjIwAziGMALAKC1gB9xBGAFilxzQNIyOAEwgjAKwSzSLkEMAdhBEAVgnxXBrAOYQRAFaJhhE6aQB3EEYAWCW66RkjI4A7CCMArBKbpmFkBHAGYQSAVaIjI3TSAO4gjACwSih87E+maQB3EEYAWIUFrIB7CCMArNK9gNVwIQAGDB93AFaJrRlhZARwBmEEgFWYpgHcQxgBYBX2GQHcQxgBYBW6aQD3EEYAWKV7msZwIQAGDGEEgFWYpgHcQxgBYBUWsALuIYwAsEqIkRHAOYQRAFaJEEYA5xBGAFgl2k3jYZoGcAZhBIBVomtGEskigDMIIwCsQjcN4B7CCACr0E0DuIcwAsAqjIwA7jmjMFJVVaXMzEwlJycrLy9PO3fuPO35lZWV+spXvqLhw4crIyNDK1as0H/+858zKhjA0BZbM0IYAZwRdxipqamR3+9XeXm5GhsblZWVpcLCQh06dKjX8zdv3qy77rpL5eXl+uijj/TMM8+opqZGq1atOuviAQw9TNMA7ok7jKxfv17FxcUqKirS1KlTVV1drREjRmjTpk29nv/OO+9o9uzZuummm5SZmalrrrlGN9544+eOpgBwE9M0gHviCiNdXV1qaGhQQUFB9zdISFBBQYHq6+t7vWbWrFlqaGiIhY99+/Zp27Zt+vrXv37K+3R2dioYDPZ4AXDD8YERRkYAhwyL5+TW1laFQiH5fL4ex30+n3bv3t3rNTfddJNaW1t15ZVXKhKJ6L///a+WLFly2mmaiooKrVmzJp7SAAwRPLUXcE+/d9Ps2LFDa9eu1RNPPKHGxkb9+te/1tatW3X//fef8prS0lK1tbXFXs3Nzf1dJgBLME0DuCeukZG0tDQlJiYqEAj0OB4IBJSent7rNffcc48WLlyo2267TZJ02WWXqaOjQ7fffrvuvvtuJSScnIe8Xq+8Xm88pQEYImIjI4QRwBlxjYwkJSUpJydHdXV1sWPhcFh1dXXKz8/v9ZpPP/30pMCRmJgoqfuBWAAQ1b0dPGEEcEVcIyOS5Pf7tXjxYuXm5mrmzJmqrKxUR0eHioqKJEmLFi3SuHHjVFFRIUmaN2+e1q9fr+nTpysvL0979+7VPffco3nz5sVCCQBEMU0DuCfuMLJgwQIdPnxYZWVlamlpUXZ2tmpra2OLWpuamnqMhKxevVoej0erV6/WgQMHdPHFF2vevHn6yU9+cu5+CgBDRvSpvXTTAO7wRAbBXEkwGFRqaqra2tqUkpJiuhwA/ajq93v18Ot79O3c8XroW1mmywFwFvr6+5tn0wCwCtvBA+4hjACwSnTNCNM0gDsIIwCsEubZNIBzCCMArBKimwZwDmEEgFXopgHcQxgBYJXufUYMFwJgwPBxB2AVtoMH3EMYAWAVtoMH3EMYAWAVtoMH3EMYAWCVEK29gHMIIwCscjyLEEYAhxBGAFglHKabBnANH3cAVoluekY3DeAOwggAq4TppgGcQxgBYBW2gwfcQxgBYBW6aQD3EEYAWIV9RgD3EEYAWIXt4AH3EEYAWCX61F4WsALuIIwAsEok2tpLFgGcQRgBYBX2GQHcQxgBYBWe2gu4hzACwCp00wDuIYwAsArdNIB7CCMArBKmmwZwDmEEgFW6t4M3XAiAAcPHHYBV2A4ecA9hBIBVIixgBZxDGAFgldg+I4yMAM4gjACwSnQ7eLppAHcQRgBYJcymZ4BzCCMArNK9HbzhQgAMGD7uAKzCyAjgHsIIAKuE6KYBnEMYAWAVtoMH3EMYAWAVpmkA9xBGAFjleBZhmgZwCGEEgFWia0YYGAHcQRgBYJXYNA0jI4AzCCMArBLrpmFoBHAGYQSAVeimAdxDGAFgFbppAPcQRgBYhU3PAPcQRgBYJcxTewHnEEYAWIUFrIB7CCMArBLmqb2Ac/i4A7BGJBLR8SyiBEZGAGcQRgBYI9rWKzFNA7iEMALAGtH1IhILWAGXEEYAWCPaSSPR2gu4hDACwBonjowwTQO4gzACwBonrhmhmwZwBx93ANYIs4AVcBJhBIA1ekzTsGYEcAZhBIA1ohueeTySh5ERwBmEEQDWiD2XhiACOOWMwkhVVZUyMzOVnJysvLw87dy587Tn/+tf/1JJSYnGjBkjr9erL3/5y9q2bdsZFQxg6OK5NICbhsV7QU1Njfx+v6qrq5WXl6fKykoVFhZqz549Gj169Ennd3V16Wtf+5pGjx6tF198UePGjdM//vEPXXDBBeeifgBDSHQBK500gFviDiPr169XcXGxioqKJEnV1dXaunWrNm3apLvuuuuk8zdt2qQjR47onXfe0XnnnSdJyszMPLuqAQxJ0dZeRkYAt8T174+uri41NDSooKCg+xskJKigoED19fW9XvPqq68qPz9fJSUl8vl8mjZtmtauXatQKHTK+3R2dioYDPZ4ARj6QrEn9hJGAJfEFUZaW1sVCoXk8/l6HPf5fGppaen1mn379unFF19UKBTStm3bdM8992jdunV64IEHTnmfiooKpaamxl4ZGRnxlAlgkIpO09DWC7il32dmw+GwRo8erQ0bNignJ0cLFizQ3Xffrerq6lNeU1paqra2ttirubm5v8sEYAEWsAJuimvNSFpamhITExUIBHocDwQCSk9P7/WaMWPG6LzzzlNiYmLs2JQpU9TS0qKuri4lJSWddI3X65XX642nNABDQCjMNA3gorhGRpKSkpSTk6O6urrYsXA4rLq6OuXn5/d6zezZs7V3716FT3gc58cff6wxY8b0GkQAuCu6AStZBHBL3NM0fr9fGzdu1HPPPaePPvpId9xxhzo6OmLdNYsWLVJpaWns/DvuuENHjhzRsmXL9PHHH2vr1q1au3atSkpKzt1PAWBIoJsGcFPcrb0LFizQ4cOHVVZWppaWFmVnZ6u2tja2qLWpqUkJJ2wSkJGRoddff10rVqzQ5ZdfrnHjxmnZsmVauXLlufspAAwJdNMAbvJEIic8mcpSwWBQqampamtrU0pKiulyAPSTd/cf0beq6zXhohF688dzTZcD4Cz19fc3+xwCsAbTNICbCCMArME0DeAmwggAa0Sb7hgZAdxCGAFgDUZGADcRRgBYo3s7eMOFABhQfOQBWCMcHRlhmgZwCmEEgDVi28ETRgCnEEYAWCM6MsJTewG3EEYAWCNENw3gJMIIAGt0d9MYLgTAgOIjD8Aa3d00jIwALiGMALAGC1gBNxFGAFgjxAJWwEmEEQDWiD5EnAWsgFsIIwCsEe2m8RBGAKcQRgBYo3uaxnAhAAYUH3kA1qCbBnATYQSANeimAdxEGAFgDbaDB9xEGAFgjejICN00gFsIIwCs0b0dPGEEcAlhBIA1woyMAE4ijACwxvEswsgI4BjCCABrdHfTGC4EwIAijACwBt00gJsIIwCswT4jgJsIIwCswVN7ATcRRgBYg+3gATcRRgBYI/rUXqZpALcQRgBYI8xTewEn8ZEHYA22gwfcRBgBYI0w28EDTiKMALBGLIwwMgI4hTACwBohumkAJxFGAFiDbhrATYQRANagmwZwEx95ANZgO3jATYQRANZgO3jATYQRANZgO3jATYQRANZgmgZwE2EEgDWOZxFGRgDHEEYAWKN70zPDhQAYUIQRANZgmgZwE2EEgDXCdNMATiKMALAG28EDbiKMALAG0zSAmwgjAKzBNA3gJsIIAGswMgK4iTACwBoh9hkBnEQYAWCNCPuMAE4ijACwRmyahjQCOIUwAsAasdZe1owATiGMALAG3TSAmwgjAKxBNw3gJsIIAGvw1F7ATYQRANbo3g7ecCEABtQZfeSrqqqUmZmp5ORk5eXlaefOnX26bsuWLfJ4PJo/f/6Z3BbAEMc0DeCmuMNITU2N/H6/ysvL1djYqKysLBUWFurQoUOnvW7//v360Y9+pKuuuuqMiwUwtLGAFXBT3GFk/fr1Ki4uVlFRkaZOnarq6mqNGDFCmzZtOuU1oVBIN998s9asWaNLLrnkrAoGMHSFI4yMAC6KK4x0dXWpoaFBBQUF3d8gIUEFBQWqr68/5XX33XefRo8erVtvvbVP9+ns7FQwGOzxAjD0hcLH/iSMAG6JK4y0trYqFArJ5/P1OO7z+dTS0tLrNW+//baeeeYZbdy4sc/3qaioUGpqauyVkZERT5kABimmaQA39eua9fb2di1cuFAbN25UWlpan68rLS1VW1tb7NXc3NyPVQKwBd00gJuGxXNyWlqaEhMTFQgEehwPBAJKT08/6fy//e1v2r9/v+bNmxc7Fg4fG4cdNmyY9uzZo0mTJp10ndfrldfrjac0AENAmG4awElx/fsjKSlJOTk5qqurix0Lh8Oqq6tTfn7+SedPnjxZ77//vnbt2hV7feMb39DcuXO1a9cupl8A9BBimgZwUlwjI5Lk9/u1ePFi5ebmaubMmaqsrFRHR4eKiookSYsWLdK4ceNUUVGh5ORkTZs2rcf1F1xwgSSddBwA2GcEcFPcYWTBggU6fPiwysrK1NLSouzsbNXW1sYWtTY1NSkhgQlfAPFjASvgJk8kcvzTb7FgMKjU1FS1tbUpJSXFdDkA+smkVdsUCkf0p1VflS8l2XQ5AM5SX39/M4QBwBrRaRpmaQC3EEYAWCHaSSNJiaQRwCmEEQBWCJ0wY8yaEcAthBEAVgidMDKSQBgBnEIYAWCFcIRpGsBVhBEAVjhxZIRpGsAthBEAVjj+pAhJbHoGuIYwAsAKLGAF3EUYAWCFE9eMkEUAtxBGAFghfMKGZx6maQCnEEYAWCH2xF6CCOAcwggAK8Se2MscDeAcwggAK0S7aRgZAdxDGAFghdg0DSMjgHMIIwCsEJumIYsAziGMALBCmJERwFmEEQBWiI6MEEYA9xBGAFghOjLCVvCAewgjAKwQ7aYhjADuIYwAsALdNIC7CCMArNC96ZnhQgAMOD72AKwQZjt4wFmEEQBWYDt4wF2EEQBWiD61l5ERwD2EEQBWYAEr4C7CCAArdG8HTxgBXEMYAWCF4wMjjIwADiKMALACD8oD3EUYAWCF6JoRumkA9xBGAFiBbhrAXYQRAFZgZARwF2EEgBVCjIwAziKMALBCmH1GAGcRRgBYIRQ+9ifTNIB7CCMArNC9gNVwIQAGHGEEgBWYpgHcRRgBYIVoN42HBayAcwgjAKzAPiOAuwgjAKwQa+1lmgZwDmEEgBVCxx+URzcN4B7CCAAr0E0DuIswAsAKbAcPuIswAsAKbAcPuIswAsAKYRawAs4ijACwwvEswj4jgIMIIwCsEIrtwGq4EAADjo89ACuw6RngLsIIACvQTQO4izACwAqMjADuIowAsALbwQPuIowAsALTNIC7CCMArMA0DeAuwggAKzAyAriLMALACtFNz8gigHsIIwCswDQN4C7CCAArRLtpmKYB3HNGYaSqqkqZmZlKTk5WXl6edu7cecpzN27cqKuuukqjRo3SqFGjVFBQcNrzAbipezt4wgjgmrjDSE1Njfx+v8rLy9XY2KisrCwVFhbq0KFDvZ6/Y8cO3Xjjjfr973+v+vp6ZWRk6JprrtGBAwfOungAQwfTNIC74g4j69evV3FxsYqKijR16lRVV1drxIgR2rRpU6/nP//88/re976n7OxsTZ48WU8//bTC4bDq6urOungAQ0couoCVkRHAOXGFka6uLjU0NKigoKD7GyQkqKCgQPX19X36Hp9++qmOHj2qCy+88JTndHZ2KhgM9ngBGNq6R0YMFwJgwMUVRlpbWxUKheTz+Xoc9/l8amlp6dP3WLlypcaOHdsj0PxfFRUVSk1Njb0yMjLiKRPAIMR28IC7BrSb5sEHH9SWLVv08ssvKzk5+ZTnlZaWqq2tLfZqbm4ewCoBmBBm0zPAWcPiOTktLU2JiYkKBAI9jgcCAaWnp5/22kceeUQPPvigfve73+nyyy8/7bler1derzee0gAMcrEwwgJWwDlxjYwkJSUpJyenx+LT6GLU/Pz8U1730EMP6f7771dtba1yc3PPvFoAQ1aIbhrAWXGNjEiS3+/X4sWLlZubq5kzZ6qyslIdHR0qKiqSJC1atEjjxo1TRUWFJOmnP/2pysrKtHnzZmVmZsbWlpx//vk6//zzz+GPAmAwo5sGcFfcYWTBggU6fPiwysrK1NLSouzsbNXW1sYWtTY1NSkhoXvA5cknn1RXV5e+9a1v9fg+5eXluvfee8+uegBDRqybhn2hAefEHUYkaenSpVq6dGmvf7djx44eX+/fv/9MbgHAMbHt4JmmAZzDv0EAWIHt4AF3EUYAWIHt4AF3EUYAWCHEPiOAswgjAKxwfGCEkRHAQYQRAFaITtMk8H8lwDl87AFYgW4awF2EEQBWCNNNAziLMALACmwHD7iLMALACnTTAO4ijACwQvd28IQRwDWEEQBWiI2MME0DOIcwAsAK4fCxPxkZAdxDGAFghVg3DSMjgHMIIwCsEO2mIYsA7iGMALAC+4wA7iKMALBCiG4awFmEEQBWYDt4wF2EEQBWiD21l5ERwDmEEQBWYDt4wF2EEQBW6N4O3nAhAAYcH3sAVmA7eMBdhBEAVmDTM8BdhBEAxkUikdgCVg9hBHAOYQSAcdEgIjFNA7iIMALAuNAJaYRpGsA9hBEAxkXXi0h00wAu4mMPwLgeIyNM0wDOIYwAMC504sgI0zSAcwgjAIwLMzICOI0wAsA4FrACbiOMADDuxNZesgjgHsIIAOOi3TQJHjY9A1xEGAFgXIjn0gBOI4wAMC4aRuikAdxEGAFgXOwheYyMAE4ijAAwLjZNw8gI4CTCCADjYgtYGRkBnEQYAWBcKHzsT6ZpADcRRgAYxwJWwG2EEQDGnbjPCAD3EEYAGEc3DeA2wggA45imAdxGGAFgHCMjgNsIIwCMo5sGcBthBIBx3dM0hgsBYARhBIBxTNMAbiOMADCOBayA2wgjAIxjZARwG2EEgHHdm54RRgAXEUYAGBftpuFBeYCbCCMAjIuuGUkkiwBOIowAMI41I4DbCCMAjKObBnAbYQSAcYyMAG4jjAAwLrZmhDACOIkwAsA4pmkAt51RGKmqqlJmZqaSk5OVl5ennTt3nvb8F154QZMnT1ZycrIuu+wybdu27YyKBTA0HZ+lYWQEcFTcYaSmpkZ+v1/l5eVqbGxUVlaWCgsLdejQoV7Pf+edd3TjjTfq1ltv1Xvvvaf58+dr/vz5+uCDD866eABDQyjCg/IAl8UdRtavX6/i4mIVFRVp6tSpqq6u1ogRI7Rp06Zez//Zz36ma6+9Vj/+8Y81ZcoU3X///ZoxY4Yef/zxsy4ewNDANA3gtmHxnNzV1aWGhgaVlpbGjiUkJKigoED19fW9XlNfXy+/39/jWGFhoV555ZVT3qezs1OdnZ2xr4PBYDxl9tkzb/9d//PPT/vlewPou48+OfYZZ5oGcFNcYaS1tVWhUEg+n6/HcZ/Pp927d/d6TUtLS6/nt7S0nPI+FRUVWrNmTTylnZGt//+gGpv+1e/3AdA3qcPPM10CAAPiCiMDpbS0tMdoSjAYVEZGxjm/zzdzxit/0kXn/PsCiF9SYqJuyB1vugwABsQVRtLS0pSYmKhAINDjeCAQUHp6eq/XpKenx3W+JHm9Xnm93nhKOyM3503o93sAAIDTi2sBa1JSknJyclRXVxc7Fg6HVVdXp/z8/F6vyc/P73G+JG3fvv2U5wMAALfEPU3j9/u1ePFi5ebmaubMmaqsrFRHR4eKiookSYsWLdK4ceNUUVEhSVq2bJmuvvpqrVu3Ttdff722bNmid999Vxs2bDi3PwkAABiU4g4jCxYs0OHDh1VWVqaWlhZlZ2ertrY2tki1qalJCQndAy6zZs3S5s2btXr1aq1atUqXXnqpXnnlFU2bNu3c/RQAAGDQ8kQi0b0P7RUMBpWamqq2tjalpKSYLgcAAPRBX39/82waAABgFGEEAAAYRRgBAABGEUYAAIBRhBEAAGAUYQQAABhFGAEAAEYRRgAAgFGEEQAAYFTc28GbEN0kNhgMGq4EAAD0VfT39udt9j4owkh7e7skKSMjw3AlAAAgXu3t7UpNTT3l3w+KZ9OEw2EdPHhQI0eOlMfjOWffNxgMKiMjQ83NzTzzpg94v+LD+xUf3q++472KD+9XfM7l+xWJRNTe3q6xY8f2eIju/zUoRkYSEhI0fvz4fvv+KSkp/AcaB96v+PB+xYf3q+94r+LD+xWfc/V+nW5EJIoFrAAAwCjCCAAAMMrpMOL1elVeXi6v12u6lEGB9ys+vF/x4f3qO96r+PB+xcfE+zUoFrACAIChy+mREQAAYB5hBAAAGEUYAQAARhFGAACAUYSRE2zdulV5eXkaPny4Ro0apfnz55suyXqdnZ3Kzs6Wx+PRrl27TJdjpf379+vWW2/VxIkTNXz4cE2aNEnl5eXq6uoyXZo1qqqqlJmZqeTkZOXl5Wnnzp2mS7JSRUWFrrjiCo0cOVKjR4/W/PnztWfPHtNlDQoPPvigPB6Pli9fbroUqx04cEDf+c53dNFFF2n48OG67LLL9O677/b7fQkjx7300ktauHChioqK9Je//EV/+MMfdNNNN5kuy3p33nmnxo4da7oMq+3evVvhcFhPPfWUPvzwQz366KOqrq7WqlWrTJdmhZqaGvn9fpWXl6uxsVFZWVkqLCzUoUOHTJdmnTfffFMlJSX64x//qO3bt+vo0aO65ppr1NHRYbo0q/35z3/WU089pcsvv9x0KVb75z//qdmzZ+u8887Tb3/7W/31r3/VunXrNGrUqP6/eQSRo0ePRsaNGxd5+umnTZcyqGzbti0yefLkyIcffhiRFHnvvfdMlzRoPPTQQ5GJEyeaLsMKM2fOjJSUlMS+DoVCkbFjx0YqKioMVjU4HDp0KCIp8uabb5ouxVrt7e2RSy+9NLJ9+/bI1VdfHVm2bJnpkqy1cuXKyJVXXmnk3oyMSGpsbNSBAweUkJCg6dOna8yYMbruuuv0wQcfmC7NWoFAQMXFxfrlL3+pESNGmC5n0Glra9OFF15ougzjurq61NDQoIKCgtixhIQEFRQUqL6+3mBlg0NbW5sk8d/SaZSUlOj666/v8d8Yevfqq68qNzdXN9xwg0aPHq3p06dr48aNA3Jvwoikffv2SZLuvfderV69Wq+99ppGjRqlOXPm6MiRI4ars08kEtEtt9yiJUuWKDc313Q5g87evXv12GOP6bvf/a7pUoxrbW1VKBSSz+frcdzn86mlpcVQVYNDOBzW8uXLNXv2bE2bNs10OVbasmWLGhsbVVFRYbqUQWHfvn168skndemll+r111/XHXfcoR/84Ad67rnn+v3eQzqM3HXXXfJ4PKd9RefzJenuu+/WN7/5TeXk5OjZZ5+Vx+PRCy+8YPinGDh9fb8ee+wxtbe3q7S01HTJRvX1/TrRgQMHdO211+qGG25QcXGxocoxFJSUlOiDDz7Qli1bTJdipebmZi1btkzPP/+8kpOTTZczKITDYc2YMUNr167V9OnTdfvtt6u4uFjV1dX9fu9h/X4Hg374wx/qlltuOe05l1xyiT755BNJ0tSpU2PHvV6vLrnkEjU1NfVniVbp6/v1xhtvqL6+/qTnFuTm5urmm28ekBRtg76+X1EHDx7U3LlzNWvWLG3YsKGfqxsc0tLSlJiYqEAg0ON4IBBQenq6oarst3TpUr322mt66623NH78eNPlWKmhoUGHDh3SjBkzYsdCoZDeeustPf744+rs7FRiYqLBCu0zZsyYHr8HJWnKlCl66aWX+v3eQzqMXHzxxbr44os/97ycnBx5vV7t2bNHV155pSTp6NGj2r9/vyZMmNDfZVqjr+/Xz3/+cz3wwAOxrw8ePKjCwkLV1NQoLy+vP0u0Sl/fL+nYiMjcuXNjo24JCUN6ULLPkpKSlJOTo7q6ulgrfTgcVl1dnZYuXWq2OAtFIhF9//vf18svv6wdO3Zo4sSJpkuy1le/+lW9//77PY4VFRVp8uTJWrlyJUGkF7Nnzz6pVfzjjz8ekN+DQzqM9FVKSoqWLFmi8vJyZWRkaMKECXr44YclSTfccIPh6uzzxS9+scfX559/viRp0qRJ/CutFwcOHNCcOXM0YcIEPfLIIzp8+HDs7/jXv+T3+7V48WLl5uZq5syZqqysVEdHh4qKikyXZp2SkhJt3rxZv/nNbzRy5MjYuprU1FQNHz7ccHV2GTly5Elrab7whS/ooosuYo3NKaxYsUKzZs3S2rVr9e1vf1s7d+7Uhg0bBmQklzBy3MMPP6xhw4Zp4cKF+uyzz5SXl6c33nhjYPqrMaRt375de/fu1d69e08KaxEemq0FCxbo8OHDKisrU0tLi7Kzs1VbW3vSolZITz75pCRpzpw5PY4/++yznztlCHyeK664Qi+//LJKS0t13333aeLEiaqsrNTNN9/c7/f2RPi/IQAAMIiJawAAYBRhBAAAGEUYAQAARhFGAACAUYQRAABgFGEEAAAYRRgBAABGEUYAAIBRhBEAAGAUYQQAABhFGAEAAEYRRgAAgFH/C332Jg+3GV+LAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "x = np.arange(-6, 6, 0.1)\n",
    "y = step_function(x)\n",
    "\n",
    "plt.plot(x, y)\n",
    "plt.show()\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.3Sigmoid函数和阶跃函数的比较"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjcAAAHHCAYAAABDUnkqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAABh10lEQVR4nO3dd3hTdfvH8Xc6kraUlha6gEKZArJk1QLKEERA3IriI4iKDz6Ag5+iqIC4UBygoKCIoiiCC9wgVlFkCLKRIRsEWnYLLV3J+f0RCJQOWmh70uTzuq5cOefke865E0pz9zsthmEYiIiIiHgIH7MDEBERESlJSm5ERETEoyi5EREREY+i5EZEREQ8ipIbERER8ShKbkRERMSjKLkRERERj6LkRkRERDyKkhsRERHxKEpuRLxQXFwcd999t9lhFGratGlYLBZ27tx53rLl4f2ISNlRciPiQdatW8ctt9xCzZo1CQgIoFq1anTt2pUJEyaYHZpXOHjwIA899BANGjQgMDCQyMhI2rRpw+OPP86JEydc5WbMmMH48ePNC1TEw1m0tpSIZ1i8eDGdOnWiRo0a9OvXj+joaPbs2cPSpUvZtm0bW7dudZXNzMzEx8cHf39/EyMunN1uJzs7G5vNhsViKbRsXFwcHTt2ZNq0aWUTXD6OHDnCZZddRmpqKvfccw8NGjTg8OHDrF27lu+++461a9cSFxcHwLXXXsv69euLVCslIsXnZ3YAIlIyXnjhBUJDQ1m+fDmVKlXK9dqBAwdy7dtstjKM7ML4+vri6+trdhhFNnXqVHbv3s2iRYto27ZtrtdSU1OxWq0mRSbifdQsJeIhtm3bxqWXXponsQGIjIzMtZ9fH5W1a9fSoUMHAgMDqV69Os8//zwffPBBnn4vcXFxXHvttSxYsIBWrVoRGBhIkyZNWLBgAQBfffUVTZo0ISAggJYtW7Jq1ao88fzyyy9cccUVVKhQgUqVKnH99dezcePGXGXy63NjGAbPP/881atXJygoiE6dOvH333+f97PJzs4mPDyc/v3753ktNTWVgIAAHn30UdexCRMmcOmllxIUFERYWBitWrVixowZhd5j27Zt+Pr6cvnll+d5LSQkhICAAAA6duzI999/z65du7BYLFgsFleNDjhr1UaNGkXdunWx2WzExsYybNgwMjMzc13TYrEwePBgPvnkEy655BLX5/3777+f9/MQ8XSquRHxEDVr1mTJkiWsX7+exo0bF+vcvXv30qlTJywWC8OHD6dChQq89957BdbwbN26lT59+vDf//6X//znP7z66qv06tWLyZMn8+STT/K///0PgDFjxnDbbbexefNmfHycf0v9/PPPdO/endq1a/PMM89w8uRJJkyYQLt27Vi5cmWuL/pzjRw5kueff54ePXrQo0cPVq5cydVXX01WVlah78/f358bb7yRr776infeeSdXLcqcOXPIzMzk9ttvB2DKlCk8+OCD3HLLLTz00ENkZGSwdu1a/vzzT/r06VPgPWrWrIndbmf69On069evwHJPPfUUKSkp/Pvvv4wbNw6A4OBgABwOB9dddx1//PEH999/Pw0bNmTdunWMGzeOf/75hzlz5uS61m+//casWbN48MEHsdlsvP3221xzzTUsW7as2D8DIh7FEBGP8NNPPxm+vr6Gr6+vkZCQYAwbNsyYN2+ekZWVladszZo1jX79+rn2hwwZYlgsFmPVqlWuY4cPHzbCw8MNwNixY0eucwFj8eLFrmPz5s0zACMwMNDYtWuX6/g777xjAMavv/7qOta8eXMjMjLSOHz4sOvYmjVrDB8fH6Nv376uYx988EGuex84cMCwWq1Gz549DYfD4Sr35JNPGkCu95Of0zF+++23uY736NHDqF27tmv/+uuvNy699NJCr5WfpKQkIyIiwgCMBg0aGAMHDjRmzJhhHDt2LE/Znj17GjVr1sxzfPr06YaPj4+xcOHCXMcnT55sAMaiRYtcxwADMP766y/XsV27dhkBAQHGjTfeWOz4RTyJmqVEPETXrl1ZsmQJ1113HWvWrGHs2LF069aNatWq8c033xR67ty5c0lISKB58+auY+Hh4dx55535lm/UqBEJCQmu/fj4eAA6d+5MjRo18hzfvn07APv372f16tXcfffdhIeHu8o1bdqUrl278sMPPxQY488//0xWVhZDhgzJ1cH44YcfLvS9nda5c2eqVKnCrFmzXMeOHj3K/Pnz6d27t+tYpUqV+Pfff1m+fHmRrntaVFQUa9asYeDAgRw9epTJkyfTp08fIiMjee655zCKMHbj888/p2HDhjRo0IBDhw65Hp07dwbg119/zVU+ISGBli1buvZr1KjB9ddfz7x587Db7cWKX8STKLkR8SCtW7fmq6++4ujRoyxbtozhw4dz/PhxbrnlFjZs2FDgebt27aJu3bp5jud3DMiVwACEhoYCEBsbm+/xo0ePuu4DcMkll+S5ZsOGDTl06BBpaWkFxghQr169XMcjIiIICwvL95yz+fn5cfPNN/P111+7+q989dVXZGdn50puHn/8cYKDg2nTpg316tVj0KBBLFq06LzXB4iJiWHSpEns37+fzZs38+abbxIREcHIkSOZOnXqec/fsmULf//9NxEREbke9evXB/J2DD/3swCoX78+6enpHDx4sEgxi3giJTciHshqtdK6dWtefPFFJk2aRHZ2Np9//nmJXb+gUUwFHS9KrUVZuP322zl+/Dg//vgjAJ999hkNGjSgWbNmrjINGzZk8+bNzJw5k/bt2/Pll1/Svn17Ro0aVeT7WCwW6tevz5AhQ/j999/x8fHhk08+Oe95DoeDJk2aMH/+/Hwfp/syiUjh1KFYxMO1atUKcDYJFaRmzZq55sE5Lb9jF6NmzZoAbN68Oc9rmzZtokqVKlSoUKHQc7ds2ULt2rVdxw8ePOiqGTqfK6+8kpiYGGbNmkX79u355ZdfeOqpp/KUq1ChAr1796Z3795kZWVx00038cILLzB8+HDXqKeiql27NmFhYbk+/4Lm7alTpw5r1qzhqquuOu/cPuD8LM71zz//EBQURERERLHiFPEkqrkR8RC//vprvjUkp/ux5NcUdFq3bt1YsmQJq1evdh07cuRIkWobiiMmJobmzZvz4YcfcuzYMdfx9evX89NPP9GjR48Cz+3SpQv+/v5MmDAh1/sszky/Pj4+3HLLLXz77bdMnz6dnJycXE1SAIcPH861b7VaadSoEYZhkJ2dXeC1//zzz3yb1JYtW8bhw4dzff4VKlQgJSUlT9nbbruNvXv3MmXKlDyvnTx5Ms/1lyxZwsqVK137e/bs4euvv+bqq68uV3MEiZQ01dyIeIghQ4aQnp7OjTfeSIMGDcjKymLx4sXMmjWLuLi4fOd4OW3YsGF8/PHHdO3alSFDhriGgteoUYMjR44UqRahqF555RW6d+9OQkIC9957r2soeGhoKM8880yB50VERPDoo48yZswYrr32Wnr06MGqVav48ccfqVKlSpHv37t3byZMmMCoUaNo0qQJDRs2zPX61VdfTXR0NO3atSMqKoqNGzcyceJEevbsScWKFQu87vTp0/nkk0+48cYbadmyJVarlY0bN/L+++8TEBDAk08+6SrbsmVLZs2axdChQ2ndujXBwcH06tWLu+66i88++4yBAwfy66+/0q5dO+x2O5s2beKzzz5j3rx5rpo4gMaNG9OtW7dcQ8EBRo8eXeTPQ8QjmTpWS0RKzI8//mjcc889RoMGDYzg4GDDarUadevWNYYMGWIkJyfnKnvuUHDDMIxVq1YZV1xxhWGz2Yzq1asbY8aMMd58800DMJKSknKd27Nnzzz3B4xBgwblOrZjxw4DMF555ZVcx3/++WejXbt2RmBgoBESEmL06tXL2LBhQ64y5w4FNwzDsNvtxujRo42YmBgjMDDQ6Nixo7F+/fp8309BHA6HERsbawDG888/n+f1d955x7jyyiuNypUrGzabzahTp47x2GOPGSkpKYVed+3atcZjjz1mtGjRwggPDzf8/PyMmJgY49ZbbzVWrlyZq+yJEyeMPn36GJUqVTKAXMPCs7KyjJdfftm49NJLDZvNZoSFhRktW7Y0Ro8enSuG05/3xx9/bNSrV8+w2WzGZZddlmvYvYi30tpSIlKghx9+mHfeeYcTJ06omcPNWCwWBg0axMSJE80ORcTtqM+NiADOPh1nO3z4MNOnT6d9+/ZKbESkXFGfGxEBnBPCdezYkYYNG5KcnMzUqVNJTU1lxIgRZocmIlIsSm5EBIAePXrwxRdf8O6772KxWGjRogVTp07lyiuvNDs0EZFiUZ8bERER8SjqcyMiIiIeRcmNiIiIeBSv63PjcDjYt28fFStWLNGJyURERKT0GIbB8ePHqVq1Kj4+hdfNeF1ys2/fvjwrF4uIiEj5sGfPHqpXr15oGa9Lbk5Pn75nzx5CQkJMjkZERESKIjU1ldjY2EKXQTnN65Kb001RISEhSm5ERETKmaJ0KVGHYhEREfEoSm5ERETEoyi5EREREY/idX1uisput5OdnW12GJIPq9V63mGAIiLivZTcnMMwDJKSkjh27JjZoUgBfHx8qFWrFlar1exQRETEDSm5OcfpxCYyMpKgoCBN9OdmTk/CuH//fmrUqKF/HxERyUPJzVnsdrsrsalcubLZ4UgBIiIi2LdvHzk5Ofj7+5sdjoiIuBl1XDjL6T42QUFBJkcihTndHGW3202ORERE3JGSm3yoqcO96d9HREQKo+RGREREPIqpyc3vv/9Or169qFq1KhaLhTlz5pz3nAULFtCiRQtsNht169Zl2rRppR6niIiIlB+mJjdpaWk0a9aMt956q0jld+zYQc+ePenUqROrV6/m4Ycf5r777mPevHmlHGn5cPDgQR544AFq1KiBzWYjOjqabt26sWjRIoAiJ5AiIiLlmamjpbp370737t2LXH7y5MnUqlWL1157DYCGDRvyxx9/MG7cOLp161ZaYZYbN998M1lZWXz44YfUrl2b5ORkEhMTOXz4sNmhiZgqK8fBgeMZBb4eGuhPxQDnyLuMbDuHTmQWWDYk0J+QU2Uzc+wcPF5w2YoB/oQGOstm2x0kpxYcQ7DNj0pBzs7yOXYHSYWUrWD1I6zCqY71DoP9KScLLBtk9SP8VFmHw2BfIWUD/X2pHGxz7f97NL3AsjY/XyIqnim799hJDMPIt6zVz4fIigGu/X3HTuIoqKyvD5EhZ8ompWSQ43DkW9bPx4fo0DNlk1MzyLbnX9bXx0JMaKBr/0BqBlkFlPWxWKha6ayyxzPIysm/rMViodpZZQ+dyCQju+DBDtUqBbr6DR4+kcnJQspWDQ3Ex8dZ9khaFulZOQWWjQkNxPdU2WPpWZzILLhsVEgA/r7Ouo2U9GyOZxY8YW1kxQCsfqfKnszmeEbBZSMq2rD5+Rb4elkqV0PBlyxZQpcuXXId69atGw8//HCB52RmZpKZeeaXT2pqammFZ6pjx46xcOFCFixYQIcOHQCoWbMmbdq0ASAuLg6AG2+80fXazp07Afj6668ZPXo0GzZsoGrVqvTr14+nnnoKPz/nj4fFYuHtt9/mm2++YcGCBcTExDB27FhuueWWsn2TIhcgx+6g67jf2HW44C/qZ3o14u52tQBYs+cYvd9dWmDZx69pwAMd6wDwT9IJek38o8CyD3auy9CrLwFg1+E0urz+e4Fl72tfi6evbQRA8vFM2r/8a4Fl74yvwQs3NgEg9WR2oWVvuqwar/duDkCW3VFo2WsujWbyXS1d+1eM/ZUCchA61I/gw3vauPavfv030rLy/6JuExfOZwMTXPvXTfyDQyey8i3buFoI3w25wrV/6zuL2XMk/4SsTkQFEv+vo2u/79RlbE4+nm/ZqqEBLB5+lWt/wPQVrNlzLN+ylYL8WT3yatf+Q5+uZsn2/P9ItPn5sPn5M3+kP/7FWhI3Hci3LMD2F3twekzEyG/+5vu1+wssu350N4Jtzt/DY37YyOcr/i2w7PKnuriSzXHz/+HDJbsKLPv7Y52oUdk5Kvjt37byzm/bCyw79+EraBAdAsC0RTsZ9/M/APhix0o2VnJcz1PubEKjyECwZ4JfIEQ2KPC6pa1cJTdJSUlERUXlOhYVFUVqaionT54kMDAwzzljxoxh9OjRF33vwjJmH4uFAH/fEi0bZC3eP01wcDDBwcHMmTOHyy+/HJvNluv15cuXExkZyQcffMA111yDr68zhoULF9K3b1/efPNNrrjiCrZt28b9998PwKhRo1znjxgxgpdeeok33niD6dOnc/vtt7Nu3ToaNmxYrDhFzPBe31a8+MNGFm/L/wvq9F+8AD4+Fmx+BbfY+51V1mKh0LK+uZYJKfy6vr5nXZfCr3v6r+7TCo3XN/fowkKv65f3ugUlN3li8Pclx5F/YX+/c2PwLTAO6znXtfr6FFz2nFoCq19hZc+Jt5Drnnvcv5Dr5ilbyHXP5X+en7Wz+RXjun4+FkL8cggmnWBOUoEMgsigguUkQWQSvCEJ/LMh6wQd/91DjDWZQDIJJAsbWQSSic2SRSBZ1PrMD4wsyMnk/pNp3GfLwEY2fpZ8arK+PGs79nK417wuIxajoHrEMmaxWJg9ezY33HBDgWXq169P//79GT58uOvYDz/8QM+ePUlPT883ucmv5iY2NpaUlBRCQkJylc3IyGDHjh3UqlWLgICAXK/FPfF9gXF1uiSCD/qf+Qum4Yi5BVY1xtcKZ9Z/z/wF0+K5+RxJy/sXzM6XehZ4v4J8+eWXDBgwgJMnT9KiRQs6dOjA7bffTtOmTYH8P+MuXbpw1VVX5fpMP/74Y4YNG8a+fftc5w0cOJBJkya5ylx++eW0aNGCt99+u9hxXqzC/p1ERDxG9klIPwzpR5zPJ484tzOOwclTj4xjkJHifGSmQkaq89lR8B/ZJc8CfgHgawU/q/O56mVw+yclepfU1FRCQ0Pz/f4+V7mquYmOjiY5OTnXseTkZEJCQvJNbABsNlueWgxPdfPNN9OzZ08WLlzI0qVL+fHHHxk7dizvvfced999d77nrFmzhkWLFvHCCy+4jtntdjIyMkhPT3dNaJiQkJDrvISEBFavXl1ab0VExDPZs+FEMhxPguP7nc8nDjiPnTgAaQcg7SCkHYbstIu/n7Ui2ILBGgzWCmCrCP5Bzm1rEPiffj79CHA++wWAf6Dz2S8A/Gx5932tp5Ia90sl3C+iQiQkJPDDDz/kOjZ//vw8X7ylYcOzBXdY9jlnUrkVI7oUUDJv2T8e73RxgZ0jICCArl270rVrV0aMGMF9993HqFGjCkxuTpw4wejRo7npppvyvZZIeZeSns3UP7Zj8/dlUKe6Zocjni4jBY7uhGN7IGUPHNvtfE7dByl7nUkMxWgw8fGHoHAIqgyB4RBYCQLDTj0qQUAlCAg99RwCthBnAmOr6ExofIrWlOVpTE1uTpw4wdatW137O3bsYPXq1YSHh1OjRg2GDx/O3r17+eijjwAYOHAgEydOZNiwYdxzzz388ssvfPbZZ3z/fcFNRiWlOH1gSqvshWjUqJFr+Le/v3+eJQtatGjB5s2bqVu38F/6S5cupW/fvrn2L7vsshKPV6SkHTuZxZu/bKWCVcmNlJCMVDi8NffjyA44ugNOHj3/+T5+EBwNFU89gqNOPSKgQiRUiIAKVZwPWwhoVvZiMzW5+euvv+jU6UzNxdChQwHo168f06ZNY//+/ezevdv1eq1atfj+++955JFHeOONN6hevTrvvfeehoEDhw8f5tZbb+Wee+6hadOmVKxYkb/++ouxY8dy/fXXA84RU4mJibRr1w6bzUZYWBgjR47k2muvpUaNGtxyyy34+PiwZs0a1q9fz/PPP++6/ueff06rVq1o3749n3zyCcuWLWPq1KlmvV2RIrOf6uTq46MvCCmmrDQ4sAmS18PBTacemyF1b+HnBVWBSjVOPWIhNBZCqkFoNedzUBWvrVEpK6YmNx07dixwXgQg39mHO3bsyKpVq0oxqvIpODiY+Ph4xo0bx7Zt28jOziY2NpYBAwbw5JNPAvDaa68xdOhQpkyZQrVq1di5cyfdunXju+++49lnn+Xll1/G39+fBg0acN999+W6/ujRo5k5cyb/+9//iImJ4dNPP6VRo0ZmvFWRYjk9n4qvkhspzMmjsH8N7FsN+1dD0jo4vI0Cm5AqREKVelC5DlSuC+G1ISzO+bBVLLOwJX9uM1qqrBTW21qjcPJXlJFsZUn/TlIc/yQf5+pxvxNewcrKEV3NDkfcgcPurI35dznsWe58PrIt/7IVIiHqUohs5Jy3JaIBVKnv7O8iZcpjR0uJiBSXq1lK/Ra8lz0b9q6EXX/ArsWwZ5lzuPS5wuIgpjnENHM+optAcGRZRyslQMmNiHi008mNr7o4eA/DcPaN2fYLbF8AuxZB1oncZWwhUL0VVG/tfFRr6RyVJB5ByY2cl5e1XIqHcfW5Uc2NZ8tKhx2/w5afYMt8SNmd+/XAcIhrDzXbQc0EiGoMPu6xDpKUPCU3IuLR6kYG883gdupQ7IlOHnMmMxu/ga2JkH3W+mF+AVCzLdTuBLU7nkpmVH3nLZTciIhHC7L60bR6JbPDkJKSeQI2/wjrPnc2OznOWqU6NBbqXQ31u0HcFc6Zd8UrKbkRERH35rDD9l9h9aew+YfcNTQRDaBhL2hwrbMTsJofBSU3IuLh9h47yTer91E52MptrWLNDkeK48gOWD3D+Uj998zx8NrQ+BZocgtEXGJefOK2lNyIiEfbeSiNl+duon5UsJKb8sDhgK3zYdkU2Pozrkn0AipB09ug2e1QtYVqaKRQSm5ExKNpnptyIiMVVn4Ey6c4F548rU5nuOwuuKSHc8VqkSJQ13EvYrFYXItommnBggVYLBaOHTtWYJlp06ZRqVKlMotJPJeWX3Bzqfth/igY1xh+esqZ2ASEQsJgGLIS7poNjW9SYiPFopobD3Lw4EFGjhzJ999/T3JyMmFhYTRr1oyRI0fSrl079u/fT1hYmNlh0rZtW/bv309oaKjZoYgXUHLjpo7ugoWvOfvTnB7xVKU+JAyCJrdppJNcFCU3HuTmm28mKyuLDz/8kNq1a5OcnExiYiKHDx8GIDo62uQInaxWq9vEIp7P7nA+q1nKTRzb7UxqVn0MjhznsRoJ0PZBqH+N5qKREqGfIg9x7NgxFi5cyMsvv0ynTp2oWbMmbdq0Yfjw4Vx33XVA3mapxYsX07x5cwICAmjVqhVz5szBYrGwevVq4Ezz0bx587jssssIDAykc+fOHDhwgB9//JGGDRsSEhJCnz59SE8/MzQzMzOTBx98kMjISAICAmjfvj3Lly93vZ5fs9S0adOoUaMGQUFB3Hjjja6ETORinVl+QcmNqU4chB8egzdbwIppzsSmdkfoPxfumQsNeiixkRKjmpvzMYzccyqUJf+gIo8ICA4OJjg4mDlz5nD55Zdjs9kKLZ+amkqvXr3o0aMHM2bMYNeuXTz88MP5ln3mmWeYOHEiQUFB3Hbbbdx2223YbDZmzJjBiRMnuPHGG5kwYQKPP/44AMOGDePLL7/kww8/pGbNmowdO5Zu3bqxdetWwsPzrt3y559/cu+99zJmzBhuuOEG5s6dy6hRo4r0vkXOR8svmCwrDZa+DX+8AVnHncdqd4QOTziXQRApBUpuzic7HV6sas69n9wH1gpFKurn58e0adMYMGAAkydPpkWLFnTo0IHbb7+dpk2b5ik/Y8YMLBYLU6ZMISAggEaNGrF3714GDBiQp+zzzz9Pu3btALj33nsZPnw427Zto3bt2gDccsst/Prrrzz++OOkpaUxadIkpk2bRvfu3QGYMmUK8+fPZ+rUqTz22GN5rv/GG29wzTXXMGzYMADq16/P4sWLmTt3btE+J5FCxNcKZ+b9lxNs06+7MmUYzlmE54+E4/udx2KaQdfnoHYHc2MTj6c6QA9y8803s2/fPr755huuueYaFixYQIsWLZg2bVqesps3b6Zp06YEBJwZgdCmTZt8r3t2chQVFUVQUJArsTl97MCBAwBs27aN7OxsVzIE4O/vT5s2bdi4cWO+19+4cSPx8fG5jiUk6C86KRmVg21cXrsyjaupA3uZSVoPH/SArwY4E5tKNeDmqTBggRIbKRP6U+Z8/IOcNShm3buYAgIC6Nq1K127dmXEiBHcd999jBo1irvvvvvCw/D3d21bLJZc+6ePORyOC76+iHiIzBPwy/Ow7F0w7OAXCFc+6hzWraHcUoaU3JyPxVLkpiF31KhRo3zntrnkkkv4+OOPyczMdPXPObvT74WqU6cOVquVRYsWUbNmTQCys7NZvnx5gX16GjZsyJ9//pnr2NKlSy86FhGALcnHWbztMNXDArmqYZTZ4XiuLT/Ddw9Dyh7nfsProNuLUEmzQkvZU7OUhzh8+DCdO3fm448/Zu3atezYsYPPP/+csWPHcv311+cp36dPHxwOB/fffz8bN25k3rx5vPrqq4CzJuZCVahQgQceeIDHHnuMuXPnsmHDBgYMGEB6ejr33ntvvuc8+OCDzJ07l1dffZUtW7YwceJE9beRErNy91FGffM3n/y52+xQPFP6EZg9ED652ZnYVKoB//kSek9XYiOmUc2NhwgODiY+Pp5x48a5+r3ExsYyYMAAnnzyyTzlQ0JC+Pbbb3nggQdo3rw5TZo0YeTIkfTp0ydXP5wL8dJLL+FwOLjrrrs4fvw4rVq1Yt68eQVOIHj55ZczZcoURo0axciRI+nSpQtPP/00zz333EXFIQJwaiS45rkpDVt/hjn/gxPJgAUufwA6PQW2YLMjEy9nMYxT4yS9RGpqKqGhoaSkpBASEpLrtYyMDHbs2EGtWrUu+gu+PPrkk0/o378/KSkpBAYGmh1Ogbz930mK5+Olu3h6znq6XRrFO3e1Mjscz5B90rlkwrJ3nPtVLoHr34LY1ubGJR6tsO/vc6nmxot99NFH1K5dm2rVqrFmzRoef/xxbrvtNrdObESKS8svlLCk9fDlvXBwk3O/zX+h62jw1+8NcR9KbrxYUlISI0eOJCkpiZiYGG699VZeeOEFs8MSKVFaFbwErZwOPzwKORlQIRJueBvqdTU7KpE8lNx4sWHDhrkmzhPxVFp+oQRkpTuXTlj9sXO/ble4cTJUqGJuXCIFUHIjIh5Nyy9cpCPbYdZdkLweLD7Q6Ulo/39aB0rcmpKbfHhZH+tyR/8+UhzdG8dQL7IikSGFr7cm+djxO3zWF04ehQoRzlmGNcOwlANKbs5yeubd9PR0dap1Y1lZWQD4+vqaHImUB7HhQcSGF3+2b6+3fCr8OMy5enfVFnD7JxBi0jp7IsWk5OYsvr6+VKpUybVOUlBQ0EVNaCclz+FwcPDgQYKCgvDz04+vSImz58DcJ2D5FOd+k1vhugkaDSXlir4dzhEdHQ3gSnDE/fj4+FCjRg0lnlIk6/emsGF/KnUjg2lRI/+JJOWUrHT4oj/8c2qG8KtGQvuhzmVoRMoRJTfnsFgsxMTEEBkZSXZ2ttnhSD6sVis+6swoRfTT30m8+ctW+ibUVHJTmLTD8Glv+Hc5+AXATVOg0XVmRyVyQZTcFMDX11d9OkQ8gN3QPDfndXQXfHwzHN4CAZWgzyyocbnZUYlcMCU3IuLR7A7ns5KbAhzcDB9dD8f3Q0h156KXkQ3Mjkrkoii5ERGPdmb5BZMDcUdJ652JTfohiGgId32lEVHiEZTciIhHcy2/oBmKc9u3Cqbf6JzDJqYZ3DUHgsLNjkqkRCi5ERGP5lp+Qc1SZ+xZDh/fBJmpUK2VsykqsJLZUYmUGCU3IuLRtCr4OfaucNbYZB2HGm3hzs/AVtHsqERKlJIbEfFot7WKJb5WZepGBpsdivmS1sP0m5yJTc32zsTGWsHsqERKnJIbEfFojauF0rhaqNlhmO/gP87OwxnHoHob6DNTiY14LI0fEBHxdEe2w0fXOUdFxTSDOz9XU5R4NNXciIhHW7X7KP8ePUmjqiHUifDCpqkTB5x9bI7vdw73/s9sdR4Wj6eaGxHxaNOX7GLIp6v4eUOy2aGUvcwT8MmtcHQnVKoJfb+GCpXNjkqk1Cm5ERGPZvfW0VL2bPisL+xfDUGV4a7ZUDHK7KhEyoSSGxHxaKemufGu5RcMA74ZAtsSwT8I+nwOleuYHZVImVFyIyIezeHwwpqbBS/Bmk/B4gu3fgjVW5odkUiZUnIjIh7N65ZfWPcF/PaSc7vXeKh/tanhiJhByY2IeDRXnxtvaJb6dwV8Pci53XYItOhrbjwiJlFyIyIe7UyzlMmBlLaUvTCzD+RkQP1roMtosyMSMY3muRERj3Zv+1p0bxJDy5phZodSerLSYeYdcCIJIhvBTVPAx9fsqERMo+RGRDxa27pVzA6hdBkGfPcI7F/jHPJ9x0wICDE7KhFTeXpFrYiIZ/trKqydeWZkVFhNsyMSMZ1qbkTEo63YdYSUk9k0rhpKZEiA2eGUrD3L4ccnnNtdnoFaV5gajoi7UM2NiHi0l+du5p5pf7F851GzQylZJw46ZyB2ZEPD65yjo0QEUHIjIh7OI0dLOezw5T1wfB9UqQ83vA3eMNRdpIg86b+7iEgep+e58ajlFxa+Bjt+B/8K0PtjsFU0OyIRt6LkRkQ8msctv7BrMSwY49y+dhxEXGJuPCJuyPTk5q233iIuLo6AgADi4+NZtmxZoeXHjx/PJZdcQmBgILGxsTzyyCNkZGSUUbQiUt64am48IblJPwJf3geGA5rdAc16mx2RiFsyNbmZNWsWQ4cOZdSoUaxcuZJmzZrRrVs3Dhw4kG/5GTNm8MQTTzBq1Cg2btzI1KlTmTVrFk8++WQZRy4i5YXD4Xwu98svGAZ8PRhS90J4HejxqtkRibgtU5Ob119/nQEDBtC/f38aNWrE5MmTCQoK4v3338+3/OLFi2nXrh19+vQhLi6Oq6++mjvuuOO8tT0i4r0cntLnZtkU2Pw9+Frh1g/AFmx2RCJuy7TkJisrixUrVtClS5czwfj40KVLF5YsWZLvOW3btmXFihWuZGb79u388MMP9OjRo8D7ZGZmkpqamushIt5jcOe6PHv9pdSJrGB2KBfu4GaYP8K53fVZiGlmbjwibs60SfwOHTqE3W4nKioq1/GoqCg2bdqU7zl9+vTh0KFDtG/fHsMwyMnJYeDAgYU2S40ZM4bRo7WAnIi3urZpVbNDuDj2bPjqfueCmHW7QPxAsyMScXumdygujgULFvDiiy/y9ttvs3LlSr766iu+//57nnvuuQLPGT58OCkpKa7Hnj17yjBiEZGL9PsrsH81BFSC6yZqPhuRIjCt5qZKlSr4+vqSnJyc63hycjLR0dH5njNixAjuuusu7rvvPgCaNGlCWloa999/P0899RQ+PnlzNZvNhs1mK/k3ICLlwl87j5Bld9CkWigVA/zNDqd4/l0Bv5/qOHzt6xASY248IuWEaTU3VquVli1bkpiY6DrmcDhITEwkISEh33PS09PzJDC+vr4AGKc6DYqInO2hmavpM+VPth9MMzuU4slKh9n3g2GHxrdA45vNjkik3DB14cyhQ4fSr18/WrVqRZs2bRg/fjxpaWn0798fgL59+1KtWjXGjHFOWNWrVy9ef/11LrvsMuLj49m6dSsjRoygV69eriRHRORs9vI6iV/is3B4K1SMgR6vmB2NSLlianLTu3dvDh48yMiRI0lKSqJ58+bMnTvX1cl49+7duWpqnn76aSwWC08//TR79+4lIiKCXr168cILL5j1FkTEzZXL5Rd2L4U/Jzu3r58IQeHmxiNSzlgML2vPSU1NJTQ0lJSUFEJCQswOR0RKWcvn5nM4LYt5D1/JJdHlYA2m7AyY3B4Ob4HmdzoXxRSRYn1/l6vRUiIixXW65qbcrAr++1hnYhMcBd1UKy1yIcrLf3cRkQtyeuFMS3loltq/Bv4Y79zu+RoEhpkajkh5peRGRDzaqdzG/deWsmfD14Oco6MaXQ8Ne5kdkUi5ZWqHYhGR0vbo1fXJzHEQVsFqdiiFW/IWJK1z1tZoUUyRi6LkRkQ82t3tapkdwvkd3QULXnJuX/0CBEeaG49IOadmKRERMxkG/DgMck5CzfbQvI/ZEYmUe6q5ERGPtnL3UXwsFi6tGoK/Ow6Z2vQ9/DMXfPydSyy4e98gkXLADf+ni4iUDMMwuOntxdzw1iJST2abHU5emcedtTYA7R6EiEvMjUfEQyi5ERGPdXrpBXDT5RcWvASpe6FSTbjiUbOjEfEYSm5ExGPZz5qA3cfdkpvkv2HpJOd2z9fBGmRuPCIeRMmNiHgsh+PMtlvNc2MY8MMw55w2Da+Del3MjkjEoyi5ERGP5Ti75sadkpu/Z8OuP8AvQEssiJQCJTci4rFyN0uZGMjZstLgpxHO7faPQKUa5sYj4oHc5b+7iEiJc5zdodhdam7+GA+p/0JoDWj3kNnRiHgkzXMjIh7L5ufLI13q4zAM9xgtdWQHLHrDud3tBfAPNDceEQ+l5EZEPFag1ZeHutQzO4wzfnoa7JlQq4MWxhQpRWqWEhEpCzt+h03fgcUXuo/VTMQipUg1NyLisbJyHGw7eAJ/Xwt1IyuaF4jDDvOedG63ugciG5gXi4gXUM2NiHis5NQMur+xkF4TFpkbyJqZkLQObKHQ8QlzYxHxAkpuRMRjnV5+wdTOxFlp8Mtzzu0rH4UKVcyLRcRLKLkREY91ep4bUwdKLZ4Ax/c714+K/6+JgYh4DyU3IuKxDMPkmpvU/WeGfncdDX42c+IQ8TJKbkTEY9lPrS1l2tILvz4P2ekQGw+NbjAnBhEvpORGRDzW6T43pqwIfmAjrJ7h3L76BQ39FilDSm5ExGOdXjjTlKUXEp8Fw+Fc9Tu2ddnfX8SLaZ4bEfFYVYJtDOxQh2Cbb9neeNcS2PyDc8K+q0aW7b1FRMmNiHiu6NAAnuhexhPmGQb8PMq53eIuqOJGyz+IeAk1S4mIlKTNP8CeP8EvEDpowj4RM6jmRkQ8Vka2neTUDGx+vkSHBpT+De05zr42AJc/ACExpX9PEclDNTci4rFW7j5Kh1cW8J+pf5bNDdd8Cgc3QWAYtHuobO4pInkouRERj+U4Nc9NmYyWysmE3152brcfCoGVSv+eIpIvJTci4rFODwUvk3luVnwIKXugYgy0GVD69xORAim5ERGPZXctv1DKN8pKh4WvOrevfBT8A0v5hiJSGCU3IuKxHI4ymsRv+XtwIhkq1YDL+pbuvUTkvJTciIjHKpPlFzJS4Y9xzu0OT4CftfTuJSJFouRGRDxWmSy/8OdkOHkEKteDpr1L7z4iUmSa50ZEPFZseBD9EmoSGx5UOjdIPwKLJzi3Ow0HX/1KFXEH+p8oIh7r0qqhjL4+tPRusOQtyEyFqMbQ6MbSu4+IFIuapURELkT6EfjzHed2x+Hgo1+nIu5CNTci4rFOZtk5npmNzc+X0ED/kr34krcg6zhEN4EGPUv22iJyUfSnhoh4rG/X7qPNC4k8NHNVyV747FqbDk9AWcyALCJFpuRGRDyWUVqjpZZMVK2NiBtTciMiHst+am2pEp3nRrU2Im5PyY2IeCx7adTcLJkIWSdUayPixpTciIjHci2/UFI1N+eOkFKtjYhbUnIjIh6rxJdfWDrpTK3NJT1K5poiUuKU3IiIxzqz/EIJXCwj5UytzZWPqdZGxI1pnhsR8Vj1oipya8vqtKgZdvEXW/YuZKZARANo0OvirycipUbJjYh4rA71I+hQP+LiL5R5Apa87dy+4lHNRizi5vQ/VETkfFZ84Fz5O7w2XKo1pETcnWpuRMRjZWTbyXEYWH19sPpd4N9y2Sdh0ZvO7fZDtfK3SDmgmhsR8ViTFmyj8ah5PPvd3xd+kZXTIe0AhMZCs9tLLjgRKTVKbkTEY50eLeVzoSObcrJg0XjndvuHwbeEF98UkVKh5EZEPJZrnpsLTW7WzoLUvRAcBc3/U4KRiUhpUnIjIh7LtfzChUzi57CfqbVJGAz+ASUXmIiUKiU3IuKxLmr5hY3fwuGtEFAJWvUv2cBEpFSZnty89dZbxMXFERAQQHx8PMuWLSu0/LFjxxg0aBAxMTHYbDbq16/PDz/8UEbRikh54loVvLjNUoYBC19zbsf/F2wVSzYwESlVpo5pnDVrFkOHDmXy5MnEx8czfvx4unXrxubNm4mMjMxTPisri65duxIZGckXX3xBtWrV2LVrF5UqVSr74EXE7bmWXyjun3HbEiFpLfgHQfzAkg9MREqVqcnN66+/zoABA+jf31nlO3nyZL7//nvef/99nnjiiTzl33//fY4cOcLixYvx93eOWoiLiyvLkEWkHLm0agg9m8ZwSXRI8U5c+LrzuWV/CAov+cBEpFSZ1iyVlZXFihUr6NKly5lgfHzo0qULS5Ysyfecb775hoSEBAYNGkRUVBSNGzfmxRdfxG63F3ifzMxMUlNTcz1ExDvc2iqWt/q04LpmVYt+0u6lsGsR+PhDwqDSC05ESo1pyc2hQ4ew2+1ERUXlOh4VFUVSUlK+52zfvp0vvvgCu93ODz/8wIgRI3jttdd4/vnnC7zPmDFjCA0NdT1iY2NL9H2IiIf5Y5zzudntEFrN3FhE5IKY3qG4OBwOB5GRkbz77ru0bNmS3r1789RTTzF58uQCzxk+fDgpKSmux549e8owYhExk91hYHcYGKf63pxX8gb4Zy5ggXYPl2ZoIlKKTOtzU6VKFXx9fUlOTs51PDk5mejo6HzPiYmJwd/fH19fX9exhg0bkpSURFZWFlarNc85NpsNm81WssGLSLkw7Iu1fLnyXx6/pgEPdKxz/hMWveF8btgLqtQt3eBEpNSYVnNjtVpp2bIliYmJrmMOh4PExEQSEhLyPaddu3Zs3boVh8PhOvbPP/8QExOTb2IjIt6tWKOlju2B9V84t9s/XGoxiUjpM7VZaujQoUyZMoUPP/yQjRs38sADD5CWluYaPdW3b1+GDx/uKv/AAw9w5MgRHnroIf755x++//57XnzxRQYNUqc/EcmrWMsvLHkLHDkQdwVUa1nKkYlIaTJ1KHjv3r05ePAgI0eOJCkpiebNmzN37lxXJ+Pdu3fj43Mm/4qNjWXevHk88sgjNG3alGrVqvHQQw/x+OOPm/UWRMSNOYq6/EL6EVj5oXO7/SOlHJWIlDZTkxuAwYMHM3jw4HxfW7BgQZ5jCQkJLF26tJSjEhFPUOTkZtkUyE6H6KZQp3MZRCYipalcjZYSESmOIjVLZaXDsnec2+0eggtdQVxE3IaSGxHxWKfXliq05mbVx5B+GCrVhEY3lElcIlK6TG+WEhEpLc2qh2IYBtUqBeZfwJ4DSyY6t9sOAV/9ShTxBPqfLCIea8hV9QovsPFrOLYLgipD8zvLJigRKXVqlhIR72QYsOhN53ab+8EaZG48IlJiLii5qV27NocPH85z/NixY9SuXfuigxIRKXU7F8L+1eAXCK0HmB2NiJSgC0pudu7cme9K3JmZmezdu/eigxIRKQl3Tf2TS57+kR/X7c/74ulam8vuhAqVyzYwESlVxepz880337i2582bR2hoqGvfbreTmJhIXFxciQUnInIxMrMdZOY4yLNsZvLfsHU+WHwgQTOci3iaYiU3N9xwAwAWi4V+/frles3f35+4uDhee+21EgtORORi2I0C5rlZPMH53PA6CFdTuoinKVZyc3rBylq1arF8+XKqVKlSKkGJiJSE05P45ZrnJmUvrPvcud3uQROiEpHSdkFDwXfs2FHScYiIlLh8VwX/c7Jzgcya7bVApoiHuqDk5tlnny309ZEjR15QMCIiJSnP8gsZqbBimnNbtTYiHuuCkpvZs2fn2s/OzmbHjh34+flRp04dJTci4hZO5TZnkpuVH0JmKlS5BOp2NS8wESlVF5TcrFq1Ks+x1NRU7r77bm688caLDkpEpCQ0qx5KaKAfYUFWsGfD0knOF9oOBh/NYSriqSyGYeQZJXmh1q1bR69evdi5c2dJXbLEpaamEhoaSkpKCiEhIWaHIyJlZe1n8NUAqBAJD68D/wCzIxKRYijO93eJ/umSkpJCSkpKSV5SROTiGQYsPjVpX/x/ldiIeLgLapZ68803c+0bhsH+/fuZPn063bt3L5HARERKzI7fIGkd+AdBq3vMjkZEStkFJTfjxo3Lte/j40NERAT9+vVj+PDhJRKYiMjF6v7GQvannGRh1YlUBLjsLggKNzssESllmudGRDxWSnoWUSe3UfHf35xLLVz+gNkhiUgZuOg+N3v27GHPnj0lEYuISImyGwb3+f7g3Gl4HYTXMjcgESkTF5Tc5OTkMGLECEJDQ4mLiyMuLo7Q0FCefvppsrOzSzpGEZELEmY/wvW+i5w7bYeYG4yIlJkLapYaMmQIX331FWPHjiUhIQGAJUuW8Mwzz3D48GEmTZpUokGKiFyI2xw/YLXYSY9uQ1D1VmaHIyJl5IKSmxkzZjBz5sxcI6OaNm1KbGwsd9xxh5IbETFf5gluMX4CIOWygQSZHI6IlJ0LSm5sNhtxcXF5jteqVQur1XqxMYmIXLxVHxNCGtsd0VBLSy2IeJML6nMzePBgnnvuOTIzM13HMjMzeeGFFxg8eHCJBScickHsObD0bQDmhdxMoM3f5IBEpCxd8NpSiYmJVK9enWbNmgGwZs0asrKyuOqqq7jppptcZb/66quSiVREpKg2fQvHdkFgOA88+DRYA82OSETK0AUlN5UqVeLmm2/OdSw2NrZEAhIRuSiGAYsnOLfbDACretuIeJsLSm4++OCDko5DRKRk7F4Ke1eArw1a32d2NCJiggvqc9O5c2eOHTuW53hqaiqdO3e+2JhERC7c6VqbZreTMGEdbcckciQty9yYRKRMXVBys2DBArKy8v6yyMjIYOHChRcdlIjIBTm0FTY7ZyR2XD6I/SkZ7EvJMDkoESlrxWqWWrt2rWt7w4YNJCUlufbtdjtz586lWrVqJRediEhxLH0LMKD+Ndgr1wO2AuBrsZgaloiUrWIlN82bN8disWCxWPJtfgoMDGTChAklFpyISJGlHYLVM5zbbYdgdxiul3wuehU9ESlPipXc7NixA8MwqF27NsuWLSMiIsL1mtVqJTIyEl9f3xIPUkTkvJZPhZwMiGkONdthZDtcL/n6qOZGxJsUK7mpWbMmAA6H4zwlRUTKUHYGLHvXud12CFgs2I2zam7ULCXiVS5oKPhHH31U6Ot9+/a9oGBERC7I2pmQfghCY6HRDQC5m6WU3Ih4lQtKbh566KFc+9nZ2aSnp2O1WgkKClJyIyJlx+GAxROd2/EDwdf5a81igQbRFXEYhpqlRLzMBSU3R48ezXNsy5YtPPDAAzz22GMXHZSISJFtmQeHt4AtFFr2cx0OCfBn7sNXmhiYiJilxMYQ1KtXj5deeilPrY6ISKk6PWlfq7vBVtHUUETEPZToAEk/Pz/27dtXkpcUESnYvytg1yLw8XM2SYmIcIHNUt98802ufcMw2L9/PxMnTqRdu3YlEpiIyHktftP53ORWCKma66WklAzumLKUQH9ffnjoChOCExGzXFByc8MNN+Tat1gsRERE0LlzZ1577bWSiEtEpHBHdsDGU39otR2S5+WsHAc7DqURZNXcWyLe5oKSm9Pz3Bw8eBAg12R+IiJlYukkMBxQ5yqIujTPy6fnudHSCyLep9h9bo4dO8agQYOoUqUK0dHRREdHU6VKFQYPHpzvSuEiIiUu/Qismu7czqfWBs7Mc+OjYeAiXqdYNTdHjhwhISGBvXv3cuedd9KwYUPAuYjmtGnTSExMZPHixYSFhZVKsCIiAPw1FbLTIaoJ1O6YbxHjVM2NchsR71Os5ObZZ5/FarWybds2oqKi8rx29dVX8+yzzzJu3LgSDVJExCU7A/7MvdRCflzNUspuRLxOsZql5syZw6uvvponsQGIjo5m7NixzJ49u8SCExHJY+1MSDsAIdWh8U0FFnM1S6nPjYjXKVbNzf79+7n00rwd905r3LgxSUlJFx2UiEi+HI4zk/Yl/A98/Qss6u/rQ83KQVSuYC2j4ETEXRQrualSpQo7d+6kevXq+b6+Y8cOwsPDSyQwEZE8Nv8Ah7c6l1poUfgadvWjKvLbY53KKDARcSfFapbq1q0bTz31FFlZWXley8zMZMSIEVxzzTUlFpyISC6nJ+1rfa+WWhCRAhW7Q3GrVq2oV68egwYNokGDBhiGwcaNG3n77bfJzMxk+vTppRWriHiz3Uthz5/ga4X4/5odjYi4sWIlN9WrV2fJkiX873//Y/jw4a6hlhaLha5duzJx4kRiY2NLJVAR8XKLTtXaNLsdKkaft/jaf4/x+JfrqBkexOS7WpZycCLiToo9Q3GtWrX48ccfOXr0KFu2bAGgbt266msjIqXn0BZnfxuAhPwn7TvXiYwcNu5PJcfuKMXARMQdXdDyCwBhYWG0adOmJGMREcnfojcAAy7pARH1i3SK5rkR8V7FXn5BRKRMpe6HtbOc2+0fKfJpp6a50Tw3Il5IyY2IuLelb4M9C2q0hdii1xY7HKq5EfFWSm5ExH2dPAZ/feDcbv9wsU7Vwpki3sstkpu33nqLuLg4AgICiI+PZ9myZUU6b+bMmVgsFm644YbSDVBEzPHX+5B1HCIbQb2ri3Wqq8+NchsRr2N6cjNr1iyGDh3KqFGjWLlyJc2aNaNbt24cOHCg0PN27tzJo48+yhVXXFFGkYpImcrOgKWTnNvtHipwgcyCWH19iKhoI1zLL4h4HYtxerIak8THx9O6dWsmTpwIgMPhIDY2liFDhvDEE0/ke47dbufKK6/knnvuYeHChRw7dow5c+YU6X6pqamEhoaSkpJCSEhISb0NESlpf70P3z0CobHw4KpC15ESEc9XnO9vU2tusrKyWLFiBV26dHEd8/HxoUuXLixZsqTA85599lkiIyO59957yyJMESlrDvtZC2QOUmIjIsVywfPclIRDhw5ht9uJiorKdTwqKopNmzble84ff/zB1KlTWb16dZHukZmZSWZmpms/NTX1guMVkTKyYQ4c2Q6BYeddIFNE5Fym97kpjuPHj3PXXXcxZcoUqlSpUqRzxowZQ2hoqOuh5SFE3JxhwMJxzu34B8Ba4YIu8+vmA9wyaTEv/rCxBIMTkfLA1JqbKlWq4OvrS3Jycq7jycnJREfnXTtm27Zt7Ny5k169ermOORzOqdX9/PzYvHkzderUyXXO8OHDGTp0qGs/NTVVCY6IO9v6MySvA2swtBlwwZc5mJrJX7uOUjHA1F9zImICU//XW61WWrZsSWJioms4t8PhIDExkcGDB+cp36BBA9atW5fr2NNPP83x48d544038k1abDYbNputVOIXkVKw8DXnc6v+EHTha9Y5tPyCiNcy/U+aoUOH0q9fP1q1akWbNm0YP348aWlp9O/fH4C+fftSrVo1xowZQ0BAAI0bN851fqVKlQDyHBeRcmjXYti9BHytcPmgi7rU6XlutPyCiPcxPbnp3bs3Bw8eZOTIkSQlJdG8eXPmzp3r6mS8e/dufHzKVdcgEblQC193Pje/E0JiLupSWn5BxHuZntwADB48ON9mKIAFCxYUeu60adNKPiARKXv718LW+WDxgXYPXvTltPyCiPdSlYiIuIc/TtXaXHoThNe+6MvZT01P6qtmKRGvo+RGRMx38B/4e45zu/0jJXJJf18LwTY/Av19S+R6IlJ+uEWzlIh4uT9eBwy4pCdEl8zggL4JcfRNiCuRa4lI+aKaGxEx15EdsPYz5/aVj5obi4h4BCU3ImKuP8aBYYe6XaBaC7OjEREPoORGRMyT8i+snuHcvnJYiV569qp/uWvqn0xbtKNErysi7k99bkTEPIveAEc2xF0BNeJL9NI7DqWzcMsh4ipf2NpUIlJ+qeZGRMxxPBlWfOjcvvKxEr/86Un8NM2NiPdRciMi5lj8JtgzITYeal1Z4pd3Lb+g7EbE6yi5EZGyd+IALJ/q3L7yMSiFifZcyy9oEj8Rr6PkRkTK3qI3IOckVGvlHCVVCuxaW0rEaym5EZGydTz5TK1Nx+GlUmsDapYS8WZKbkSkbC1+86xam6tK7TYWLPhY1Cwl4o0shnHqzxsvkZqaSmhoKCkpKYSEhJgdjoh3OZ4MbzRzJjd3fgn1SqdJSkQ8T3G+v1VzIyJlp4xqbUTEuym5EZGyUUZ9bURElNyISNn4Y1yZ1tpMW7SD+z/6i7nrk0r9XiLiXpTciEjpS/kX/jpVa9P5qTKptVm7N4WfNiSz83Baqd9LRNyLkhsRKX2/vwL2LKjZDmp3KpNbahI/Ee+l5EZESteR7bDqY+d25xFl1tfGfmocqOa5EfE+Sm5EpHQteBkcOc6ZiGsmlNltz9TclNktRcRNKLkRkdJzYBOsneXc7vRUmd7aYWj5BRFvpeRGRErPghcBAxpcC9ValOmtT68tpWYpEe+j5EZESse+VbDha8BS5rU2cFbNjToUi3gdP7MDEBEP9fMzzucmt0JUozK//eT/tMRuGEpuRLyQkhsRKXnbfoHtC8DH3zmvjQn8fH30C07ES6lZSkRKlsNxptam9X0QFmdmNCLihZTciEjJ2jAb9q8Ba0W48lHTwpj4yxYenrmKlbuPmhaDiJhDyY2IlJycLEh8zrnd7kGoUMW0UBZuOcSc1fvYd+ykaTGIiDmU3IhIyVn5IRzdARUi4fL/mRqKRkuJeC8lNyJSMjKPw28vO7c7DANbsKnhaJ4bEe+l5EZESsYf4yDtIITXgZZ3mx2Na20p1dyIeB8lNyJy8Y7tgSVvObe7Pgu+/ubGw1lrS6nmRsTrKLkRkYuX+CzkZEDN9tCgp9nRAGf63KhZSsT7KLkRkYuzdwWs+wywQLcXwE2agVx9btwjHBEpQ5rAU0QunGHAvFMzEDe7Hao2NzWcs30+MIEcu0EFm37NiXgb/a8XkQu38VvYvQT8AqHzCLOjyaVigPn9fkTEHGqWEpELk50BPz3t3G47BEKrmRuPiMgpqrkRkQuzeAIc2wUVq0K7h8yOJo+xczdxND2b/3WsQ2x4kNnhiEgZUs2NiBRfyr+w8DXn9tXPmT5hX36+Xr2PT5ft5nBaltmhiEgZU3IjIsX30wjIOQk12kLjm82OJl9afkHEeym5EZHi2bEQ/v4KLD7Q/WW3Gfp9rjPLL5gciIiUOf23F5Gis+fAj487t1v2h5im5sZTCFfNjSa6EfE6Sm5EpOj+mgoH/obAMOj8tNnRFOp0zY2apUS8j5IbESma1P2Q+Jxzu/PTEBRubjzncSq3waLkRsTrKLkRkaKZ9yRkHYdqLZ1NUm5OC2eKeC/NcyMi57f15zOdiK8dBz6+Zkd0Xon/14Ech0FERZvZoYhIGVNyIyKFyz4J3z/q3I4fCDHNzI2niCJDAswOQURMomYpESncH+Pg6A7nTMSdnjQ7GhGR81LNjYgU7OA/zuQGoPtLYKtobjzFMPrbv7Fg4ZGu9bSIpoiXUc2NiOTP4YBvhoA9C+pdDQ2vMzuiIjMMgw8W7eT9RTvIzHGYHY6IlDElNyKSv+VTYM9SsAZDz9fddibi/JweBg6a50bEGym5EZG8ju6Cn0c7t7uOhkqx5sZTTPazshsfDQUX8TpKbkQkN8OAbx+E7DSo2Q5a3mN2RMV2eukF0Dw3It5IyY2I5LbqY9i+APwC4LoJ5XLlybOTG+U2It6n/P3WEpHSk7oPfnrKud3pSahcx9x4LlCuZin1uRHxOm6R3Lz11lvExcUREBBAfHw8y5YtK7DslClTuOKKKwgLCyMsLIwuXboUWl5EisgwYM7/ICMFqraAyweZHdEFc5w1QErNUiLex/TkZtasWQwdOpRRo0axcuVKmjVrRrdu3Thw4EC+5RcsWMAdd9zBr7/+ypIlS4iNjeXqq69m7969ZRy5iIdZ/h5s/9XZHHXjO+BbfqfBCg7w45f/68DPQzvgp+RGxOtYDOOsxmkTxMfH07p1ayZOnAiAw+EgNjaWIUOG8MQTT5z3fLvdTlhYGBMnTqRv377nLZ+amkpoaCgpKSmEhIRcdPwiHuHQVpjcHnJOQvexEP9fsyMSEcmlON/fptbcZGVlsWLFCrp06eI65uPjQ5cuXViyZEmRrpGenk52djbh4eGlFaaIZ7PnwOz7nYlNrQ7QeoDZEYmIXBRT650PHTqE3W4nKioq1/GoqCg2bdpUpGs8/vjjVK1aNVeCdLbMzEwyMzNd+6mpqRcesIgn+uN12LsCbKFww9vlcnTUuVJOZvPOb9uw+vnwcJf6ZocjImWsXP8We+mll5g5cyazZ88mICD/FYDHjBlDaGio6xEbW74mIxMpVXuWwYKXnNs9X4XQ6ubGU0JST2bz9oJtTP5tm9mhiIgJTE1uqlSpgq+vL8nJybmOJycnEx0dXei5r776Ki+99BI//fQTTZs2LbDc8OHDSUlJcT327NlTIrGLlHsnj8EX94Jhh8a3QJNbzY6oxJye50ZLL4h4J1OTG6vVSsuWLUlMTHQdczgcJCYmkpCQUOB5Y8eO5bnnnmPu3Lm0atWq0HvYbDZCQkJyPUS8nmE4F8VM2Q1hcXDtuHK1dtT5nJ7nRksviHgn08d6Dh06lH79+tGqVSvatGnD+PHjSUtLo3///gD07duXatWqMWbMGABefvllRo4cyYwZM4iLiyMpKQmA4OBggoODTXsfIuXKig9g4zfg4w+3fAABnpX0n57DT3PciHgn05Ob3r17c/DgQUaOHElSUhLNmzdn7ty5rk7Gu3fvxuesDo6TJk0iKyuLW265Jdd1Ro0axTPPPFOWoYuUT8kbYO5w53aXUVCthbnxlILTzVKanVjEO5me3AAMHjyYwYMH5/vaggULcu3v3Lmz9AMS8VSZx+HzfpCTAXW7lutZiAvjapZSciPilcr1aCkRKYbTyysc+gdCqsGNkz1i2Hd+Tic3vp759kTkPNyi5kZEysDiCWf62dz2EVSoYnZEpaZuZDDfP9hefW5EvJSSGxFvsON3+HmUc7v7S1C98FGG5V2Avy+XVg01OwwRMYkqbUU8Xeo++OIeMBzQ7A5oda/ZEYmIlCrV3Ih4sqx0+PQOSDsIUU2g5+seNZ9NQfYeO8lXK/4lPNjKnfE1zQ5HRMqYam5EPJXDAXMegP2rITAcek8Ha5DZUZWJ3YfTeW3+P7z/xw6zQxEREyi5EfFUv70MG+Y4OxD3/hjCa5kdUZlxLb+gDsUiXknJjYgnWv8l/HZqQcxrx0FcO3PjKWOaxE/Euym5EfE0e5Y757MBSBgMLe4yNx4TnJnnRsmNiDdSciPiSQ5tgRm3OWcgrtcNuj5rdkSmULOUiHdTciPiKY4nwcc3wckjULUF3PI++PiaHZUp7A7ns5qlRLyTkhsRT5CRCp/cAsd2Q3htuPNzsAWbHZVp1Cwl4t00z41IeZedAbP+A0nroEIE/Ocrj15aoSjia4XzxcAEgqz6FSfijfQ/X6Q8s2fD53fDjt/Av4KzxsaLhnwXJKyClVYVws0OQ0RMomYpkfLKYYevBsA/P4JfANzxKVS9zOyoRERMp5obkfLI4YCvB8Pfs89M0le7g9lRuY0tycf5fcshqocF0u3SaLPDEZEyppobkfLG4YDvH4E1M8DiC7d+APW6mh2VW1m95xjPfbeBGX/uNjsUETGBam5EyhOHHb4ZAqs/ASxw4zvQsJfZUbmdU9PcaLSUiJdSciNSXthzYPZ/Yf0XzhqbGydD01vNjsot2bX8gohXU3IjUh7kZMGX98DGb8HHzzlBX6PrzY7KbZ2Z58bkQETEFEpuRNxd5gn4vB9s/Rl8rXDbR3BJd7OjcmtafkHEuym5EXFnJw7CjFth3yrwD3KOiqp7ldlRub3TNTdqlhLxTkpuRNzVke0w/SY4ugOCKkOfz6F6S7OjKhe0/IKId1NyI+KO/l0Bn/aGtINQqQb8ZzZUqWt2VOXGNY2jqR9VkYiKNrNDERETKLkRcTdrP4evB4E9E6Kbwp1fQMUos6MqV6qHBVE9LMjsMETEJEpuRNyFwwG/PAd/vO7cr98dbp4CtormxiUiUs4ouRFxBxkpMPsB2Py9c7/9I9B5BPj4mhtXObXu3xTW7U2hXlQwreO0gKaIt1FyI2K2pHXwWV9nB2JfK1w3AZrdbnZU5VripmTG/7yFO+NrKLkR8UJKbkTMtOoT+H4o5GRASHW47UOo3srsqMo9h4aCi3g1JTciZsg8AXMfh1UfO/frdoGbpkCQahlKgl2T+Il4NSU3ImXt37/gqwHOZiiLD3R6Etr/H/horYCSYnc4n1VzI+KdlNyIlBV7Dix8DX57GQy7sxnqxslQ6wqzI/M4Z5ZfMDkQETGFkhuRspC8Ab4ZDHtXOPcb3ww9X4PAMHPj8lCu5RfULCXilZTciJSmnCznvDW/vwqObLCFOJOapreZHZlHcy2/oGYpEa+k5EaktOxZBt8+BAc2OPfrd4drX4eQqubG5QVubxPL5bUrUzuigtmhiIgJlNyIlLQTB2D+KFgzw7kfVAV6jIVLbwLVJJSJBtEhNIgOMTsMETGJkhuRkmLPhuXvwa8vQmaq89hl/4Euz0KFyubGJiLiRZTciFwsw4CN38DPo+HINuexmObOvjWakM8UK3YdZc+RdC6tGkK9KK3NJeJtlNyIXIzdS+GnEfDvMud+UBXo/BS06Kd1oUz06bLdfLHiX4Zdc4mSGxEvpORG5ELsWQYLXoJtic59/yBIGAxth0CA+nqYzaHRUiJeTcmNSHHs/tM5Cd/ppMbi6+xX03E4hMSYG5u4aPkFEe+m5EbkfBwO+OdHWPQm7FnqPGbxheZ94Ir/g/Ba5sYnedi1cKaIV1NyI1KQzBOwdhYsnQSHtziP+VqhaW8lNW7OoZobEa+m5EbkXIe2OId0r55xZki3LRRa3wPxA6FitLnxyXk5Ti+cqeRGxCspuREByEqDDd/Aqo9h1x9njofXgdb3QYu7wKZRN+WFq8+NmqVEvJKSG/FeDgfsXgxrP4P1X0HW8VMvWKD+NdDmPqjdGXy0tHR5c1/7WvRsEkOz2EpmhyIiJlByI97FMGDfKlj/pTOhOb7vzGthcdD8P9D8DgitblqIcvHia2tGaBFvpuRGPJ/DDruXwMbvYNN3kLLnzGu2UGjUy9lJuGZ71dKIiHgAJTfimdKPwNafYctPzueTR8+85h8E9btB41ugXlfws5kXp5SKZTuOcDQ9i6bVQ4kJDTQ7HBEpY0puxDNkZ8CeP2H7Audj/2owHGdeD6gEl/SAhtdCnc7gry88TzZu/j8s2X6YN++4jOua6d9axNsouZHyKfME/Lscdi12Pvb+BTkZuctENXbWzNS7Gqq3AV/9uHsLjZYS8W76bS/uz+GAw1th7wpnQvPvMkjeAIY9d7ngaKjdEWp3gFodILSaKeGK+VxrS6kLlYhXUnIj7iUnEw5uhuS/IWkt7FvtfM46kbdsaCzUbOt81GgLVeqB/lIXztTcaPkFEe+k5EbMkZMJR3bAoc3OZObgJjiwEQ79A46cvOX9AiGmKVRvfeahmhkpwJmaGyU3It5IyY2UnuwMOLbLmcQc3QlHd8Dhbc4mpmO7cnf4PVtAKEQ1gahLoWpziGkOVeqrz4wUmavmRsmNiFfSt4VcGHsOpB2A4/shZS+knnqk/AvH9sCx3c7XC2OtCBH1IaIBRFzifI66FEKqqXlJLoprbSn9HIl4JSU3ckb2Sef8MOmHIe3gmecTB848n0g+9TgAGOe/prUihMc5Z/8NqwWV60DlulC5HgRHKomRUvHgVXU5nJZF/ahgs0MRERO4RXLz1ltv8corr5CUlESzZs2YMGECbdq0KbD8559/zogRI9i5cyf16tXj5ZdfpkePHmUYsRsyDOdQ6MwTzjWSMk89MlKdK1tnpEJGCmQcO/N88vTjqDORyTlZvHtafCE4ytn3JaSac8mCkKpQqYazs2+lGhAYpgRGytw1jWPMDkFETGR6cjNr1iyGDh3K5MmTiY+PZ/z48XTr1o3NmzcTGRmZp/zixYu54447GDNmDNdeey0zZszghhtuYOXKlTRu3NiEd1AAhx3sWc6Os/ZssGee2s7K/ZyT6UxKTj+yT556znAmG9lnP9Kcz1npzu2sNOd2VppzNNG5Q6MvhI8fBIZDhQioUBmCqjhrWIIjocKp54rRUDHG+ZqWKxARETdjMQyjCG0LpSc+Pp7WrVszceJEABwOB7GxsQwZMoQnnngiT/nevXuTlpbGd9995zp2+eWX07x5cyZPnnze+6WmphIaGkpKSgohISEl90b2LMeYfqMzabFnYSlKk00pcfhXwLBWxGENPvVcEYctBIctBN/ASgSGhENAJYzAMA7ZA3HYwnAEhuEICMOwVnTVtAT4+1Il+MzSBP8eTS/wnjY/XyIqnim779hJHAX8aFl9fYgMCXDt7085id2Rf1l/Xx+iziqblJJBjiP/jsh+Pj5Eh54peyA1gyx7/mV9LBaqVjozc+2B4xlk5eRf1mKxUO2ssgePZ5KZU3AiWT0syLV9+EQmJ7MLLlutUiCWU5/3kbQs0rPyGSl2SkxooGv0z9G0LNIKKRsdEoDfqUlejqVncSKz4LJRIQH4nyqbcjKb4xnZBZaNqGjD5ucLQGpGNqknCy5bJdhGgL+z7PGMbFKKWDYtM4ej6VkFlg2vYCXI6vy7LD0rhyNpecsu33mEsCArLWqGERLgX+C1RKT8KM73t6k1N1lZWaxYsYLhw4e7jvn4+NClSxeWLFmS7zlLlixh6NChuY5169aNOXPm5Fs+MzOTzMxM135qaurFB54fiw+WrOMFvpxp+IGvFVtAIPjayLb4s/NYDhn4k4U/mYY/mfiTgZUMrNSIDKdl3WrgF8AJw8qbv+0hnQBOGjZOYiUdG+lGAGkE0LFxTR7r1RKswZwggMbPzC8wjmubxjDx2haAc7hs6yd/AI6eeuR2VYNIpt7d2rXf+bXfCkwA2tapzIwBl7v2e7y5kGPp+X+ZNY+txJxB7Vz7N729mP0pGfmWvSSqIvMeudK13+e9pWw/mJZv2RrhQfw+rJNr/54Pl7N+b/7/3lWCbfz1dBfX/qBPVrJ8Z97PAKCC1Ze/n73Gtf9/n6/h938O5lvWxwLbx/R07T81ez1z/07KtyzApueucX2pP/fdBmav2ltg2VUjuhJWwQrAKz9tZsafuwssu/iJzq7kbeIvW3nvjx0Flv15aAfqRjr7pry3cDsTftlaYNlvB7enSfVQAD5euouxczcXWPaz/ybQplY4AF+u+Jdnvt1QYNlp/VvT8RJnTe13a/fx+JfrCiw76c4WdG/ibHb6eeMBHvx0VYFlv/pfW1rUCCvwdRHxTKYmN4cOHcJutxMVFZXreFRUFJs2bcr3nKSkpHzLJyXl/wUyZswYRo8eXTIBFybqUrbfsZB7pq8hGz+y8SMHX7LwJxtfwMKAtrV5tNslAOw/nM61434r8HL/qVWTlt0bAZB+PIMPf//V+UI+3VeaBZ3q6wKQmYPNr+CmIv9zpmwtbtmCes/kV7aga1vPOW4tTlnfkil77nH/wsqeSj7OXNdSYNlzR+f4F/LezuXnU/B1z+V/nrJnh+FbSLx5yhbjuueL16cY1z37c/OxnC+GM2V9Cylbq0oFGkRXLPA6IuK5TG2W2rdvH9WqVWPx4sUkJCS4jg8bNozffvuNP//8M885VquVDz/8kDvuuMN17O2332b06NEkJyfnKZ9fzU1sbGzJN0uJiIhIqSk3zVJVqlTB19c3T1KSnJxMdHR0vudER0cXq7zNZsNms+X7moiIiHgeU4e6WK1WWrZsSWJiouuYw+EgMTExV03O2RISEnKVB5g/f36B5UVERMS7mD4UfOjQofTr149WrVrRpk0bxo8fT1paGv379wegb9++VKtWjTFjxgDw0EMP0aFDB1577TV69uzJzJkz+euvv3j33XfNfBsiIiLiJkxPbnr37s3BgwcZOXIkSUlJNG/enLlz57o6De/evRufs+ZSadu2LTNmzODpp5/mySefpF69esyZM8e95rgRERER05g+z01ZK7V5bkRERKTUFOf7W9PLioiIiEdRciMiIiIeRcmNiIiIeBQlNyIiIuJRlNyIiIiIR1FyIyIiIh5FyY2IiIh4FCU3IiIi4lGU3IiIiIhHMX35hbJ2ekLm1NRUkyMRERGRojr9vV2UhRW8Lrk5fvw4ALGxsSZHIiIiIsV1/PhxQkNDCy3jdWtLORwO9u3bR8WKFbFYLCV67dTUVGJjY9mzZ4/WrSoCfV5Fp8+qePR5FY8+r+LR51U8JfV5GYbB8ePHqVq1aq4FtfPjdTU3Pj4+VK9evVTvERISoh/4YtDnVXT6rIpHn1fx6PMqHn1exVMSn9f5amxOU4diERER8ShKbkRERMSjKLkpQTabjVGjRmGz2cwOpVzQ51V0+qyKR59X8ejzKh59XsVjxufldR2KRURExLOp5kZEREQ8ipIbERER8ShKbkRERMSjKLkRERERj6LkphR9//33xMfHExgYSFhYGDfccIPZIbm9zMxMmjdvjsViYfXq1WaH45Z27tzJvffeS61atQgMDKROnTqMGjWKrKwss0NzG2+99RZxcXEEBAQQHx/PsmXLzA7JLY0ZM4bWrVtTsWJFIiMjueGGG9i8ebPZYZULL730EhaLhYcfftjsUNzW3r17+c9//kPlypUJDAykSZMm/PXXX2VybyU3peTLL7/krrvuon///qxZs4ZFixbRp08fs8Nye8OGDaNq1apmh+HWNm3ahMPh4J133uHvv/9m3LhxTJ48mSeffNLs0NzCrFmzGDp0KKNGjWLlypU0a9aMbt26ceDAAbNDczu//fYbgwYNYunSpcyfP5/s7Gyuvvpq0tLSzA7NrS1fvpx33nmHpk2bmh2K2zp69Cjt2rXD39+fH3/8kQ0bNvDaa68RFhZWNgEYUuKys7ONatWqGe+9957ZoZQrP/zwg9GgQQPj77//NgBj1apVZodUbowdO9aoVauW2WG4hTZt2hiDBg1y7dvtdqNq1arGmDFjTIyqfDhw4IABGL/99pvZobit48ePG/Xq1TPmz59vdOjQwXjooYfMDsktPf7440b79u1Nu79qbkrBypUr2bt3Lz4+Plx22WXExMTQvXt31q9fb3Zobis5OZkBAwYwffp0goKCzA6n3ElJSSE8PNzsMEyXlZXFihUr6NKli+uYj48PXbp0YcmSJSZGVj6kpKQA6GepEIMGDaJnz565fsYkr2+++YZWrVpx6623EhkZyWWXXcaUKVPK7P5KbkrB9u3bAXjmmWd4+umn+e677wgLC6Njx44cOXLE5Ojcj2EY3H333QwcOJBWrVqZHU65s3XrViZMmMB///tfs0Mx3aFDh7Db7URFReU6HhUVRVJSkklRlQ8Oh4OHH36Ydu3a0bhxY7PDcUszZ85k5cqVjBkzxuxQ3N727duZNGkS9erVY968eTzwwAM8+OCDfPjhh2VyfyU3xfDEE09gsVgKfZzuDwHw1FNPcfPNN9OyZUs++OADLBYLn3/+ucnvouwU9fOaMGECx48fZ/jw4WaHbKqifl5n27t3L9dccw233norAwYMMCly8QSDBg1i/fr1zJw50+xQ3NKePXt46KGH+OSTTwgICDA7HLfncDho0aIFL774Ipdddhn3338/AwYMYPLkyWVyf78yuYuH+L//+z/uvvvuQsvUrl2b/fv3A9CoUSPXcZvNRu3atdm9e3dphuhWivp5/fLLLyxZsiTPuiOtWrXizjvvLLNM32xF/bxO27dvH506daJt27a8++67pRxd+VClShV8fX1JTk7OdTw5OZno6GiTonJ/gwcP5rvvvuP333+nevXqZofjllasWMGBAwdo0aKF65jdbuf3339n4sSJZGZm4uvra2KE7iUmJibXdyBAw4YN+fLLL8vk/kpuiiEiIoKIiIjzlmvZsiU2m43NmzfTvn17ALKzs9m5cyc1a9Ys7TDdRlE/rzfffJPnn3/etb9v3z66devGrFmziI+PL80Q3UpRPy9w1th06tTJVSvo46NKWACr1UrLli1JTEx0Tb3gcDhITExk8ODB5gbnhgzDYMiQIcyePZsFCxZQq1Yts0NyW1dddRXr1q3Ldax///40aNCAxx9/XInNOdq1a5dnWoF//vmnzL4DldyUgpCQEAYOHMioUaOIjY2lZs2avPLKKwDceuutJkfnfmrUqJFrPzg4GIA6deror8h87N27l44dO1KzZk1effVVDh486HpNtRMwdOhQ+vXrR6tWrWjTpg3jx48nLS2N/v37mx2a2xk0aBAzZszg66+/pmLFiq5+SaGhoQQGBpocnXupWLFinr5IFSpUoHLlyuqjlI9HHnmEtm3b8uKLL3LbbbexbNky3n333TKrZVZyU0peeeUV/Pz8uOuuuzh58iTx8fH88ssvZTfGXzzW/Pnz2bp1K1u3bs2T/BmGYVJU7qN3794cPHiQkSNHkpSURPPmzZk7d26eTsYCkyZNAqBjx465jn/wwQfnbSIVKUzr1q2ZPXs2w4cP59lnn6VWrVqMHz+eO++8s0zubzH021BEREQ8iBrqRURExKMouRERERGPouRGREREPIqSGxEREfEoSm5ERETEoyi5EREREY+i5EZEREQ8ipIbERER8ShKbkTErdx9992udaHKyrRp06hUqVKZ3lNESo+SGxEREfEoSm5ExG117NiRBx98kGHDhhEeHk50dDTPPPNMrjIWi4VJkybRvXt3AgMDqV27Nl988YXr9QULFmCxWDh27Jjr2OrVq7FYLOzcuZMFCxbQv39/UlJSsFgsWCyWPPcQkfJFyY2IuLUPP/yQChUq8OeffzJ27FieffZZ5s+fn6vMiBEjuPnmm1mzZg133nknt99+Oxs3bizS9du2bcv48eMJCQlh//797N+/n0cffbQ03oqIlBElNyLi1po2bcqoUaOoV68effv2pVWrViQmJuYqc+utt3LfffdRv359nnvuOVq1asWECROKdH2r1UpoaCgWi4Xo6Giio6MJDg4ujbciImVEyY2IuLWmTZvm2o+JieHAgQO5jiUkJOTZL2rNjYh4HiU3IuLW/P39c+1bLBYcDkeRz/fxcf6aMwzDdSw7O7tkghMRt6TkRkTKvaVLl+bZb9iwIQAREREA7N+/3/X66tWrc5W3Wq3Y7fbSDVJEyoySGxEp9z7//HPef/99/vnnH0aNGsWyZcsYPHgwAHXr1iU2NpZnnnmGLVu28P333/Paa6/lOj8uLo4TJ06QmJjIoUOHSE9PN+NtiEgJUXIjIuXe6NGjmTlzJk2bNuWjjz7i008/pVGjRoCzWevTTz9l06ZNNG3alJdffpnnn38+1/lt27Zl4MCB9O7dm4iICMaOHWvG2xCREmIxzm6IFhEpZywWC7Nnzy7zWY1FxH2p5kZEREQ8ipIbERER8Sh+ZgcgInIx1LIuIudSzY2IiIh4FCU3IiIi4lGU3IiIiIhHUXIjIiIiHkXJjYiIiHgUJTciIiLiUZTciIiIiEdRciMiIiIeRcmNiIiIeJT/B2hfWCrDhSwkAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "x = np.arange(-6, 6, 0.1)\n",
    "y1 = step_function(x)\n",
    "y2 = sigmoid_function(x)\n",
    "\n",
    "plt.plot(x, y1, label='Step', linestyle='--')\n",
    "plt.plot(x, y2, label='Sigmoid')\n",
    "plt.xlabel(\"Input\")\n",
    "plt.ylabel(\"Output\")\n",
    "plt.title('Sigmoid vs Step')\n",
    "plt.legend()\n",
    "plt.show()\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 不同点\n",
    "\n",
    "    1. 平滑性不同。sigmoid函数是一条平滑的曲线，输出随着输入**发生连续性的变化**。而阶跃函数以0为界，输出**发生急剧性的变化**。sigmoid函数的平滑性对神经网络的学习具有重要意义。\n",
    "\n",
    "    2. 阶跃函数只能返回0或1；sigmoid函数可以返回0.731 ...、0.880 ...等实数。即：**感知机中神经元之间流动的是0或1的二元信号**，而**神经网络中流动的是连续的实数值信号**。\n",
    "\n",
    "- 共同点：\n",
    "\n",
    "    1. 二者曲线的形状相似，都是**输入小时，输出接近于0；输入大时，输出接近于1**。即：`当输入信号为重要信息时，阶跃函数和sigmoid函数都会输出较大的值；当输入信号为不重要的信息时，两者都输出较小的值。`\n",
    "\n",
    "    2. 不管输入信号有多小，或者有多大，输出信号的值都在0到1之间。\n",
    "\n",
    "\n",
    "## 2.4非线性函数和线性函数\n",
    "\n",
    "- 线性函数：$h(x)=cx$\n",
    "\n",
    "- 非线性函数：线性函数以外的函数\n",
    "\n",
    "所以，上述的两个激活函数都是非线性函数。实际上，对于神经网络来说，不可能使用线性函数作为激活函数，原因：如果使用线性函数，那么n次后的输出就是$c_nc_{n-1}...c_1x$，（$x$是初始输入），令$c_nc_{n-1}...c_1=a$，就可以概括成**一个**新激活函数，那么这样之前若干层激活函数实质上就只有一层，此时多层神经网络就变成一层神经网络（因为中间那么多层没有意义），由第一章可知，单层神经网络即单层感知机，是一个线性模型，无法解决异或问题"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.5Relu激活函数\n",
    "\n",
    "公式如下：\n",
    "\n",
    "![image-20230311224824063](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/image-20230311224824063.png)\n",
    "\n",
    "程序实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhYAAAGdCAYAAABO2DpVAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAAArLklEQVR4nO3deXxTdb7G8adQCAXasrVApUBBkKWUHQZxAUWRAQRFxuugo1yvM3rrigtWHTaVojLKqAi4jHrn6uBFRR1UkEXABWSTXQplkVJoy5qUAumS3D/SlBZZmpLknJN83q9X/khION859DV9PE9+vxPhdrvdAgAA8INqRg8AAABCB8ECAAD4DcECAAD4DcECAAD4DcECAAD4DcECAAD4DcECAAD4DcECAAD4TWSwD+hyubR//35FR0crIiIi2IcHAABV4Ha7lZ+fr4SEBFWrdu7rEkEPFvv371diYmKwDwsAAPwgKytLzZo1O+efBz1YREdHS/IMFhMTE+zDAwCAKnA4HEpMTCz7PX4uQQ8W3vojJiaGYAEAgMVc6GsMfHkTAAD4DcECAAD4DcECAAD4DcECAAD4jc/BIjs7W7fffrsaNmyoqKgoderUSWvWrAnEbAAAwGJ8WhVy9OhR9e3bV/3799fXX3+tuLg47dixQ/Xr1w/UfAAAwEJ8ChYvvPCCEhMT9e6775a9lpSU5PehAACANflUhXzxxRfq0aOHRo4cqfj4eHXt2lVvvfXWeT/jdDrlcDgqPAAAQGjyKVjs2rVLM2bMUJs2bbRgwQLdd999evDBB/X++++f8zPp6emKjY0te7CdNwAAoSvC7Xa7K/vmmjVrqkePHvrxxx/LXnvwwQe1evVqrVix4qyfcTqdcjqdZc+9W4La7XZ23gQAwCIcDodiY2Mv+PvbpysWTZs2VYcOHSq81r59e+3du/ecn7HZbGXbd7ONNwAAoc2nYNG3b19lZGRUeG379u1q0aKFX4cCAADW5FOweOSRR7Ry5UpNnjxZmZmZ+vDDD/Xmm28qNTU1UPMBAIBKenH+Nk3/NlPFJS7DZvBpuWnPnj01d+5cpaWladKkSUpKStK0adM0atSoQM0HAAAqYc2eI5qxbKfcbqlr83q6vHUjQ+bw+bbpQ4YM0ZAhQwIxCwAAqIJTRSV6/OONcrulW7o3MyxUSNwrBAAAy5u6IEO7DxWocYxNfx3c4cIfCCCCBQAAFrb21yN654fdkqT0mzsptnYNQ+chWAAAYFGnikr0+BxPBTKiWzNd066x0SMRLAAAsKq/fZOhXYcKFB9t07ghxlYgXgQLAAAsaO2vR/T29+apQLwIFgAAWEz5VSA3d71E17Y3vgLxIlgAAGAxryzcrl0HPRXI+KEdjR6nAoIFAAAWsm7vUb313S5J0uSbzFOBeBEsAACwCM8qkA1ylVYgAzqYpwLxIlgAAGARryzarp0HCxQXbdO4oeZYBXImggUAABbw896jemv56QqkXu2aBk90dgQLAABM7lRRiR4rrUCGd0nQdSasQLwIFgAAmNy0RTu082CBGtW1acKN5loFciaCBQAAJvbz3qN6c/lOSdLkm5JNW4F4ESwAADAp70ZY3grk+o5NjB7pgggWAACY1LRFO5SZd1yN6ppvI6xzIVgAAGBC67OOlVUgz9+UrPp1zF2BeBEsAAAwmfIbYQ3rkqCBFqhAvAgWAACYzKuLd2hH3nE1qltTEyxSgXgRLAAAMJENWcc0c5mnAnlueCfLVCBeBAsAAEzCWXx6I6wbOyfohmTrVCBeBAsAAEyiQgVi8o2wzoVgAQCACWzcd0wzl3nuBfLc8GQ1sFgF4kWwAADAYN4KpMTl1tDOCbohuanRI1UZwQIAAIO9tjhT23OPq2Gdmppo0QrEi2ABAICBNu2za0bZKhDrViBeBAsAAAxSvgIZktJUgzpZtwLxIlgAAGCQ15dkKiM3PyQqEC+CBQAABti0z643lnoqkGeHJ6thXZvBE/kHwQIAgCArLHaVVSCDU5rq9yFQgXgRLAAACLLXl+xQRm6+GtSpqUkhUoF4ESwAAAiizdl2TfdWIMNCpwLxIlgAABAk5SuQ33dqosEpoVOBeBEsAAAIkte/zdS2nNIKZFiy0eMEBMECAIAg2Jxt1xvfZkqSJg3rqEYhVoF4ESwAAAgwbwVSXFqBDElJMHqkgCFYAAAQYNPDoALxIlgAABBAW/bbNT0MKhAvggUAAAHiqUA2qtjl1qDkJhocQhthnQvBAgCAAHljaaZ+OeBQ/do1NGlYsiIiIoweKeAIFgAABMDW/Q69vsRbgSQrLjq0KxAvggUAAH5WVHJ6FcgNHZtoSAhuhHUuBAsAAPzsjW93amtpBfLs8PCoQLwIFgAA+NEvBxx6bckOSdLEMKpAvAgWAAD4SfkKZGDHxhoaRhWIF8ECAAA/mbF0p7bsd6heGFYgXj4FiwkTJigiIqLCo127doGaDQAAy6hQgdzYUfHRtQyeyBiRvn6gY8eOWrRo0em/INLnvwIAgJDirUCKSty6vkNj3dg5dO8FciE+p4LIyEg1adIkELMAAGBJM8tVIM/dFJ4ViJfP37HYsWOHEhIS1KpVK40aNUp79+497/udTqccDkeFBwAAoWJbjkOvUoGU8SlY9O7dW++9957mz5+vGTNmaPfu3bryyiuVn59/zs+kp6crNja27JGYmHjRQwMAYAblK5AB7cO7AvGKcLvd7qp++NixY2rRooVefvll3X333Wd9j9PplNPpLHvucDiUmJgou92umJiYqh4aAADDvb5kh6Z+s12xUTW08JGrFB8TulcrHA6HYmNjL/j7+6K+eVmvXj21bdtWmZmZ53yPzWaTzRZem4MAAEJfRk6+/r64XAUSwqHCFxe1j8Xx48e1c+dONW0afhuAAADCV/EZFciwLlQgXj4Fi8cee0zLli3Tnj179OOPP+qmm25S9erVddtttwVqPgAATGfW8l3alG1XbFQNTQ7zVSBn8qkK2bdvn2677TYdPnxYcXFxuuKKK7Ry5UrFxcUFaj4AAEwlIydf0xZtlyRNuLEDFcgZfAoWs2fPDtQcAACYXnGJS49/7KlArm0Xr+FdLjF6JNPhXiEAAFTSrOW7tHGfXTG1IjX55k5UIGdBsAAAoBIycvL190WeVSDjh3ZUYyqQsyJYAABwAd4KpLDEpWvbxevmblQg50KwAADgAt78jgqksggWAACcx/bcfE1b6KlAxlGBXBDBAgCAcygucenxOZ4K5Jp28RpBBXJBBAsAAM7hre92a8M+u6JrRWryTVQglUGwAADgLHbk5uuVhZ6NsMYN6aAmsVQglUGwAADgDMUlLj328UYVlrjU/7I43dK9mdEjWQbBAgCAM7z9/W5tyDqm6FqRSr85hQrEBwQLAADKyczL18ulFchfqUB8RrAAAKBUcYlLj87ZqMJil/pdFqeRVCA+I1gAAFDqHW8FYotUOhthVQnBAgAASZl5x/W3chVI09gogyeyJoIFACDslbjcnnuBFLt0dds4jexBBVJVBAsAQNh75/td+nkvFYg/ECwAAGEtM++4pn7jqUCeGdJeCfWoQC4GwQIAELZKXG49UVqBXNU2Tn/okWj0SJZHsAAAhK1/fL9b6/YeU11bpKZQgfgFwQIAEJZ2Hjyuqd9kSJKeGUwF4i8ECwBA2PFUIBvlLHbpyjaNdGtPKhB/IVgAAMLOuz/s1tpfj3oqkBHcC8SfCBYAgLCy6+BxvbTAU4E8Pbi9LqEC8SuCBQAgbHg2wjpdgfwHFYjfESwAAGGDCiTwCBYAgLCw+1BBWQXy1O+pQAKFYAEACHklLrcen7NBzmKXrri0kW7rRQUSKAQLAEDIe+/HPVrz61HVqVldU0awEVYgESwAACHNU4FskyQ9Nbi9mtWvbfBEoY1gAQAIWa7Se4GcKnKp76UN9cdezY0eKeQRLAAAIeu9H/do9Z7SCuRmVoEEA8ECABCS9hwq0IulFUja79srsQEVSDAQLAAAIcdVei+QU0UuXd66oUb1pgIJFoIFACDkvL9ij1btOaI6NavrBTbCCiqCBQAgpOw5VKAX5lOBGIVgAQAIGS6XW098croCYRVI8BEsAAAh439W7NGq3UdUu7QCqVaNCiTYCBYAgJDw6+ECvTDfcy+QtEHtqEAMQrAAAFiedxXIyaIS9WnVUKN6tzB6pLBFsAAAWN4/V/6qn0orkBdvoQIxEsECAGBpew+f0JSvPatAnqQCMRzBAgBgWZ5VIBt0sqhEv2vVQLdTgRiOYAEAsKz//elXrdx1RFE1quvFEZ2pQEyAYAEAsKSsIxUrkOYNqUDMgGABALAcl8utxz/eoBOFJeqd1EB3/I4KxCwIFgAAy/mgfAXCKhBTuahgMWXKFEVEROjhhx/20zgAAJxf1pETSi+tQMbecJlaNKxj8EQor8rBYvXq1Zo1a5ZSUlL8OQ8AAOfk3QjrRGGJeiU10J/6tDR6JJyhSsHi+PHjGjVqlN566y3Vr1/f3zMBAHBWH6zaqxW7DqtWjWp6iQrElKoULFJTUzV48GANGDDggu91Op1yOBwVHgAA+CrryAlN+eoXSdLYG9pRgZhUpK8fmD17ttatW6fVq1dX6v3p6emaOHGiz4MBAODldrs19pONKigsUa+WDXQnFYhp+XTFIisrSw899JA++OAD1apVq1KfSUtLk91uL3tkZWVVaVAAQPj6cNVe/bjTU4GwCsTcfLpisXbtWuXl5albt25lr5WUlGj58uV6/fXX5XQ6Vb169Qqfsdlsstls/pkWABB2so6c0OQvPRXIEwPbqWUjKhAz8ylYXHvttdq0aVOF10aPHq127dpp7NixvwkVAABcDLfbrSc/PV2B3HV5S6NHwgX4FCyio6OVnJxc4bU6deqoYcOGv3kdAICL9a9VWfoh01OBvEAFYgnsvAkAMKV9R0/o+S+3SpIeu/4yJVGBWILPq0LOtHTpUj+MAQDAaW63W09+skkFhSXq0aK+RvdNMnokVBJXLAAApvOvVVn6PvOQbJHV9NLIzqpOBWIZBAsAgKmUr0AeH0gFYjUECwCAabjdbqV9SgViZQQLAIBpzF6dpe92eCqQF29JoQKxIIIFAMAUso+d1POlG2E9dv1lahVX1+CJUBUECwCA4TyrQDbquLNY3VvU139eQQViVQQLAIDhPqICCRkECwCAobKPndRz5SqQ1lQglkawAAAYxrsK5LizWN2a16MCCQEECwCAYf5vTZaWbz+ommyEFTIIFgAAQ+w/dlLPzfNUII9e15YKJEQQLAAAQee5Hfom5TuL1bV5Pf3Xla2MHgl+QrAAAATdnDX7Tlcgt1CBhBKCBQAgqA7YT+rZeZ57gTx6XVtdGk8FEkoIFgCAoPGuAsl3FqtLIhVIKCJYAACCZs7afVqa4alApo5kI6xQRLAAAARF+QrkkQFtdWl8tMETIRAIFgCAgHO73Xrq003KP1Wszon1dM+VbIQVqggWAICA+3jtPn2bcVA1q1fT1FtSFFmdXz+hin9ZAEBA5dhPaZK3Armurdo0pgIJZQQLAEDAuN1uPTW3tAJpFksFEgYIFgCAgPlkXbaWbMtTzeqee4FQgYQ+/oUBAAGRYz+lif/eIkl6+Lo2aksFEhYIFgAAvzuzAvkzG2GFDYIFAMDvPqUCCVv8SwMA/CrXcboCeWgAFUi4IVgAAPzG7Xbr6bmb5DhVrJRmsfrLVVQg4YZgAQDwm7k/Z2vRL6UVyC1UIOGIf3EAgF/kOU5pwhenK5DLmlCBhCOCBQDgonlXgThOFavTJVQg4YxgAQC4aJ+t91QgNapHaCqrQMIa//IAgIviqUA89wJ56FoqkHBHsAAAVJmnAtks+8kidbokVvde3drokWAwggUAoMo+X79fi37JVY3qEXppJLdDB8ECAFBFefmnNL50FciD17RRuyYxBk8EMyBYAAB85tkIy1OBJF8So3v7UYHAg2ABAPDZFxv2a+HW0grkls6qQQWCUvwkAAB8Ur4CeeCaNmrflAoEpxEsAACV5q1Ajp0oUseEGN1HBYIzECwAAJXmrUAiq3k2wqICwZn4iQAAVEr5CuT+ay6lAsFZESwAABfkdrv11888FUiHpjFK7X+p0SPBpAgWAIAL+vfGA1qwhQoEF8ZPBgDgvA7mOzX+882SPBVIhwQqEJwbwQIAcE7eCuQoFQgqyadgMWPGDKWkpCgmJkYxMTHq06ePvv7660DNBgAw2LyNBzR/S44iq3nuBUIFggvx6SekWbNmmjJlitauXas1a9bommuu0bBhw7Rly5ZAzQcAMMih406NK61AUvtfqo4JsQZPBCuI9OXNQ4cOrfD8+eef14wZM7Ry5Up17NjRr4MBAIw17nNPBdKeCgQ+8ClYlFdSUqI5c+aooKBAffr08edMAACDfbnxgL7alFO6CiRFNSOpQFA5PgeLTZs2qU+fPjp16pTq1q2ruXPnqkOHDud8v9PplNPpLHvucDiqNikAICgOHXfqr6UVyH9TgcBHPkfQyy67TOvXr9dPP/2k++67T3feeae2bt16zvenp6crNja27JGYmHhRAwMAAmv851t0pKBQ7ZpE634qEPgowu12uy/mLxgwYIBat26tWbNmnfXPz3bFIjExUXa7XTExrIUGADP5cuMBpX64TtWrRejz1L5KvoSrFfBwOByKjY294O/vKn/HwsvlclUIDmey2Wyy2WwXexgAQIAdLr8KpF9rQgWqxKdgkZaWpkGDBql58+bKz8/Xhx9+qKVLl2rBggWBmg8AECTjvtiiw94K5Jo2Ro8Di/IpWOTl5elPf/qTDhw4oNjYWKWkpGjBggW67rrrAjUfACAIvt50QF9uPKDqpfcCYRUIqsqnYPHOO+8Eag4AgEEOH3fqmc88Fch9V1OB4OIQSQEgzI0vrUAuaxytB65lFQguDsECAMLY15sOaF65CsQWWd3okWBxBAsACFNHCgrLNsK67+rW6tSMCgQXj2ABAGFq/BdbdOh4odo2rksFAr8hWABAGJq/+YD+vWE/FQj8jmABAGHmSEFh2SqQv1zVSinN6hk7EEIKwQIAwsyE0gqkTXxdPTSAjbDgXwQLAAgj8zfn6AsqEAQQwQIAwsTRgkI989kmSZ4KpHNiPWMHQkgiWABAmJjwbyoQBB7BAgDCwIItOfp8/X5Vi5BeogJBABEsACDEHS0o1NNzPatA/nxVa3WhAkEAESwAIMR5KhCnLo2vq4epQBBgBAsACGHlK5CpIzurVg0qEAQWwQIAQhQVCIxAsACAEDWxtAJpHVeHCgRBQ7AAgBD0zZYcfVZuFQgVCIKFYAEAIebYiUI9XXovkHuubKVuzesbPBHCCcECAELMxH9v1cF8TwXyyHVtjR4HYYZgAQAhZOHWXM39OZsKBIYhWABAiDh2olBPzfXcC4QKBEYhWABAiJhUWoG0ogKBgQgWABACFm3N1afeCuQWKhAYh2ABABZnP1FUVoH815Wt1L0FFQiMQ7AAAIubOG+L8korkDFUIDAYwQIALGzxL7n6dB0VCMyDYAEAFmU/UaS0T6lAYC4ECwCwqEnztlKBwHQIFgBgQUu25eqTdfsUESG9dEsKFQhMg2ABABZToQK5IkndWzQweCLgNIIFAFjMs19uVa7DqVaN6ujR6y8zehygAoIFAFjIt9vy9PHa0gpkJBUIzIdgAQAWYT95ugK5uy8VCMyJYAEAFvHcvK3KcZxSEhUITIxgAQAW8G1GnuasPb0KJKomFQjMiWABACbnOFWktE88Fch/9k1Sj5ZUIDAvggUAmJy3AmnZsLYeowKByREsAMDElmbk6f/WeFeBdKYCgekRLADApBynTq8CGX15knpSgcACCBYAYFLPz/tFB+yn1KJhbT0+kAoE1kCwAAATWrb9oD5ak1W6CoQKBNZBsAAAk3GcKtKTn2yUJN11eUv1SqICgXUQLADAZCZ/SQUC6yJYAICJLN9+ULNXZ0mSXhyRoto1Iw2eCPANwQIATCL/jAqkd6uGBk8E+I5gAQAmMfmrX7TffkrNG9TWEzdQgcCafAoW6enp6tmzp6KjoxUfH6/hw4crIyMjULMBQNhYvv2g/rWqtAK5hQoE1uVTsFi2bJlSU1O1cuVKLVy4UEVFRbr++utVUFAQqPkAIOTll9sI667LW+p3VCCwMJ8i8fz58ys8f++99xQfH6+1a9fqqquu8utgABAuJn+1TdnHTlKBICRc1Hcs7Ha7JKlBA9ZYA0BVfL/jkP61aq8k6QVWgSAEVPkn2OVy6eGHH1bfvn2VnJx8zvc5nU45nc6y5w6Ho6qHBICQctxZrLGlq0Du7NNCfVpTgcD6qnzFIjU1VZs3b9bs2bPP+7709HTFxsaWPRITE6t6SAAIKelf/aLsYyeV2CBKT9zQzuhxAL+oUrC4//77NW/ePH377bdq1qzZed+blpYmu91e9sjKyqrSoAAQSn7IPKQPfvJUIC+O6Kw6NioQhAaffpLdbrceeOABzZ07V0uXLlVSUtIFP2Oz2WSz2ao8IACEmuPOYj3xsacC+RMVCEKMT8EiNTVVH374oT7//HNFR0crJydHkhQbG6uoqKiADAgAoWbK154KpFn9KI2lAkGI8akKmTFjhux2u/r166emTZuWPT766KNAzQcAIeXHzEP635XeCiSFCgQhx+cqBABQNcedxXq8tAK5/XfNdfmljQyeCPA/7hUCAEFSvgJJG9Te6HGAgCBYAEAQUIEgXBAsACDACpzFeqJ0I6xRvalAENoIFgAQYFO+3qZ9R0/qknpRSvs9FQhCG8ECAALox52H9M+Vv0ry3A69LhUIQhzBAgACpKDcRlh/7N1cfalAEAYIFgAQIC/MP12BPEUFgjBBsACAAFix87D+Z4WnAnlhBBUIwgfBAgD8zLMKZIMk6bZezXVFGyoQhA+CBQD42YvztynriLcC4V4gCC8ECwDwo5W7Duv90gpkyohOiq5Vw+CJgOAiWACAn5woPL0K5LZeibqyTZzBEwHBR7AAAD95cX6G9h45oYTYWqwCQdgiWACAH6zcdVjv/bhHkjRlRAoVCMIWwQIALtKJwmKNLb0XyH/0TNRVbalAEL4IFgBwkV6cn6FfD3sqkKcHU4EgvBEsAOAi/FSuAkmnAgEIFgBQVScLS8puh35rj0RdTQUCECwAoKpeXLBNvx4+oaaxtfT0ECoQQCJYAECVrNp95HQFcnMnxVCBAJIIFgDgs5OFJXri4w1yu6U/9GimfpfFGz0SYBoECwDw0UsLMrTn8Ak1iamlpwd3MHocwFQIFgDgg9V7jujdH3dLktJHdFJsFBUIUB7BAgAqyVOBbJTbLY3s3kz9qUCA3yBYAEAlTf0mQ7sPFahJTC09M4QKBDgbggUAVMKaPUf0jx9KK5CbqUCAcyFYAMAFnCoq0eOlFcgt3ZupfzsqEOBcCBYAcAFTF3gqkMYxNv2VCgQ4L4IFAJzH2l+P6B0qEKDSCBYAcA6nikr0+JzTFcg17RobPRJgegQLADiHlxdu1y5vBcJGWEClECwA4CzW/npUb323S1JpBVKbCgSoDIIFAJzBswrEcy+QEd2oQABfECwA4AyvLNyuXQcLFB9t0zhWgQA+IVgAQDnr9p6uQCbfRAUC+IpgAQClThWV6LE5G+RySzd3vUQDOlCBAL4iWABAqVcWna5Axg/taPQ4gCURLABApRXIcioQ4GIRLACEPc9GWJ4K5CYqEOCiECwAhL1pi3Zo58ECxUXbNH4oq0CAi0GwABDWft57VG8u3ynJU4HUq13T4IkAayNYAAhb3tuhu9zS8C4Juo4KBLhoBAsAYevvi3coM++4GtW1acKNrAIB/IFgASAsrc86plnLvBVIMhUI4CcECwBhp/wqkGFdEnR9xyZGjwSEDIIFgLDz6uId2uGtQNgIC/Arn4PF8uXLNXToUCUkJCgiIkKfffZZAMYCgMDYkHVMM0srkOdvSlb9OlQggD/5HCwKCgrUuXNnTZ8+PRDzAEDAOItP3wvkxs4JGkgFAvhdpK8fGDRokAYNGhSIWQAgoE5XIDU1kVUgQEDwHQsAYWHjvmOaucxzL5DnhneiAgECxOcrFr5yOp1yOp1lzx0OR6APCQAVeCuQEpdbQzsn6IZkKhAgUAJ+xSI9PV2xsbFlj8TExEAfEgAqeG1xprbnUoEAwRDwYJGWlia73V72yMrKCvQhAaDMpn12zShdBfLc8GQ1oAIBAirgVYjNZpPNZgv0YQDgN8pXIENSmuqG5KZGjwSEPJ+DxfHjx5WZmVn2fPfu3Vq/fr0aNGig5s2b+3U4ALgYry/JVEZuvhrWoQIBgsXnYLFmzRr179+/7PmYMWMkSXfeeafee+89vw0GABdjc7Zdbyw9XYE0rMuVUyAYfA4W/fr1k9vtDsQsAOAXhcWusgpkcKemGtSJCgQIFvaxABByXl+yQ9tyPBXIpGFUIEAwESwAhJTN2XZNL61AnqUCAYKOYAEgZJxZgfyeCgQIOoIFgJDhrUAa1KmpiVQggCEIFgBCQoUKZFiyGlGBAIYgWACwvPIVyKDkJhqcQgUCGIVgAcDyXv82s6wCeXZ4stHjAGGNYAHA0jZn2/XGt57dgCcN60gFAhiMYAHAsgqLXXr8440q9lYgrAIBDEewAGBZ07/N1C8HHKpfu4YmDUtWRESE0SMBYY9gAcCStuy3a3pZBZKsuGgqEMAMCBYALKeoxKXH5ngqkIEdG2sIq0AA0yBYALCcN77dWVaBPDe8ExUIYCIECwCWsnW/Q68t2SFJmkgFApgOwQKAZXgqkA1lFchQKhDAdAgWACxjxtKd2nrAoXq1a+jZ4awCAcyIYAHAEn45UK4CubGj4qNrGTwRgLMhWAAwPW8FUlTi1vUdGuvGzglGjwTgHAgWAExv5tKd2rLfodioGnruJioQwMwIFgBM7ZcDDr1KBQJYBsECgGkVlbj0+MeeCuS6Do01rAsVCGB2BAsApjVr2U5tzvZUIM+zCgSwBIIFAFPaluPQ3xd7KpAJN3ZQfAwVCGAFBAsAplN+FciA9o01vMslRo8EoJIIFgBM583lu8oqkMmsAgEshWABwFQycvI1bdF2SVQggBURLACYRnGFCiSeCgSwIIIFANOYtXyXNmXbFVMrUs/fxO3QASsiWAAwhYycfP19kWcVyPihHdWYCgSwJIIFAMMVl26EVVji0rXt4nVzNyoQwKoIFgAM9+Z3u7Rxn6cCmXwzFQhgZQQLAIbakZuvaQupQIBQQbAAYBjvKpDCEpeuoQIBQgLBAoBh3vputzbssyu6VqQmswoECAkECwCG2JGbr1cWejbCGj+0o5rEUoEAoYBgASDoiktceuzjjSoscan/ZXEaQQUChAyCBYCge/v73dqQdUzRtSKVfnMKFQgQQggWAIIqMy9fL5dWIOOGdKACAUIMwQJA0JS43HpszkYVFrvU77I43dK9mdEjAfAzggWAoHn7u11an3VM0bZIpbMRFhCSCBYAgiIzL19/K61A/jq0g5rGRhk8EYBAIFgACLjyFcjVbeM0kgoECFkECwAB9873VCBAuCBYAAiozLzjmvqNpwJ5Zkh7JdSjAgFCGcECQMCUuNye26EXu3RV2zj9oUei0SMBCDCCBYCA+cf3u/Xz3mOqa4vUFCoQICxUKVhMnz5dLVu2VK1atdS7d2+tWrXK33MBsLidB49r6jcZkqRnBlOBAOHC52Dx0UcfacyYMRo/frzWrVunzp07a+DAgcrLywvEfAAsqMTl1uNzNshZ7NKVbRrp1p5UIEC48DlYvPzyy7rnnns0evRodejQQTNnzlTt2rX1j3/8IxDzAbCgd3/YrXXeCmQE9wIBwkmkL28uLCzU2rVrlZaWVvZatWrVNGDAAK1YseKsn3E6nXI6nWXPHQ5HFUc9v5e/yVC+szggfzeAynO7pX+t2itJenpwe11CBQKEFZ+CxaFDh1RSUqLGjRtXeL1x48batm3bWT+Tnp6uiRMnVn3CSpq9Okt5+c4LvxFAUFzZppH+gwoECDs+BYuqSEtL05gxY8qeOxwOJSb6//9s7urbUgVcsQBMwRZZXaN6N6cCAcKQT8GiUaNGql69unJzcyu8npubqyZNmpz1MzabTTabreoTVtJ/97s04McAAADn59OXN2vWrKnu3btr8eLFZa+5XC4tXrxYffr08ftwAADAWnyuQsaMGaM777xTPXr0UK9evTRt2jQVFBRo9OjRgZgPAABYiM/B4tZbb9XBgwc1btw45eTkqEuXLpo/f/5vvtAJAADCT4Tb7XYH84AOh0OxsbGy2+2KiYkJ5qEBAEAVVfb3N/cKAQAAfkOwAAAAfkOwAAAAfkOwAAAAfkOwAAAAfkOwAAAAfkOwAAAAfkOwAAAAfkOwAAAAfhPw26afybvRp8PhCPahAQBAFXl/b19ow+6gB4v8/HxJUmJiYrAPDQAALlJ+fr5iY2PP+edBv1eIy+XS/v37FR0drYiICL/9vQ6HQ4mJicrKyuIeJJXA+fIN56vyOFe+4Xz5hvPlG3+eL7fbrfz8fCUkJKhatXN/kyLoVyyqVaumZs2aBezvj4mJ4YfNB5wv33C+Ko9z5RvOl284X77x1/k635UKL768CQAA/IZgAQAA/CZkgoXNZtP48eNls9mMHsUSOF++4XxVHufKN5wv33C+fGPE+Qr6lzcBAEDoCpkrFgAAwHgECwAA4DcECwAA4DcECwAA4DchGyy+/PJL9e7dW1FRUapfv76GDx9u9Eim53Q61aVLF0VERGj9+vVGj2NKe/bs0d13362kpCRFRUWpdevWGj9+vAoLC40ezTSmT5+uli1bqlatWurdu7dWrVpl9EimlJ6erp49eyo6Olrx8fEaPny4MjIyjB7LEqZMmaKIiAg9/PDDRo9iWtnZ2br99tvVsGFDRUVFqVOnTlqzZk1Qjh2SweKTTz7RHXfcodGjR2vDhg364Ycf9Mc//tHosUzviSeeUEJCgtFjmNq2bdvkcrk0a9YsbdmyRa+88opmzpypp556yujRTOGjjz7SmDFjNH78eK1bt06dO3fWwIEDlZeXZ/RoprNs2TKlpqZq5cqVWrhwoYqKinT99deroKDA6NFMbfXq1Zo1a5ZSUlKMHsW0jh49qr59+6pGjRr6+uuvtXXrVv3tb39T/fr1gzOAO8QUFRW5L7nkEvfbb79t9CiW8tVXX7nbtWvn3rJli1uS++effzZ6JMt48cUX3UlJSUaPYQq9evVyp6amlj0vKSlxJyQkuNPT0w2cyhry8vLcktzLli0zehTTys/Pd7dp08a9cOFC99VXX+1+6KGHjB7JlMaOHeu+4oorDDt+yF2xWLdunbKzs1WtWjV17dpVTZs21aBBg7R582ajRzOt3Nxc3XPPPfrnP/+p2rVrGz2O5djtdjVo0MDoMQxXWFiotWvXasCAAWWvVatWTQMGDNCKFSsMnMwa7Ha7JPGzdB6pqakaPHhwhZ8x/NYXX3yhHj16aOTIkYqPj1fXrl311ltvBe34IRcsdu3aJUmaMGGCnnnmGc2bN0/169dXv379dOTIEYOnMx+326277rpL9957r3r06GH0OJaTmZmp1157TX/5y1+MHsVwhw4dUklJiRo3blzh9caNGysnJ8egqazB5XLp4YcfVt++fZWcnGz0OKY0e/ZsrVu3Tunp6UaPYnq7du3SjBkz1KZNGy1YsED33XefHnzwQb3//vtBOb5lgsWTTz6piIiI8z68/bckPf300xoxYoS6d++ud999VxEREZozZ47B/yuCp7Ln67XXXlN+fr7S0tKMHtlQlT1f5WVnZ+uGG27QyJEjdc899xg0OUJBamqqNm/erNmzZxs9iillZWXpoYce0gcffKBatWoZPY7puVwudevWTZMnT1bXrl315z//Wffcc49mzpwZlOMH/bbpVfXoo4/qrrvuOu97WrVqpQMHDkiSOnToUPa6zWZTq1attHfv3kCOaCqVPV9LlizRihUrfrOPfI8ePTRq1KigJVyjVfZ8ee3fv1/9+/fX5ZdfrjfffDPA01lDo0aNVL16deXm5lZ4PTc3V02aNDFoKvO7//77NW/ePC1fvlzNmjUzehxTWrt2rfLy8tStW7ey10pKSrR8+XK9/vrrcjqdql69uoETmkvTpk0r/A6UpPbt2+uTTz4JyvEtEyzi4uIUFxd3wfd1795dNptNGRkZuuKKKyRJRUVF2rNnj1q0aBHoMU2jsufr1Vdf1XPPPVf2fP/+/Ro4cKA++ugj9e7dO5Ajmkplz5fkuVLRv3//sqth1apZ5sJfQNWsWVPdu3fX4sWLy5Z3u1wuLV68WPfff7+xw5mQ2+3WAw88oLlz52rp0qVKSkoyeiTTuvbaa7Vp06YKr40ePVrt2rXT2LFjCRVn6Nu372+WLm/fvj1ovwMtEywqKyYmRvfee6/Gjx+vxMREtWjRQi+99JIkaeTIkQZPZz7Nmzev8Lxu3bqSpNatW/NfT2eRnZ2tfv36qUWLFpo6daoOHjxY9mf8V7k0ZswY3XnnnerRo4d69eqladOmqaCgQKNHjzZ6NNNJTU3Vhx9+qM8//1zR0dFl30OJjY1VVFSUwdOZS3R09G++e1KnTh01bNiQ76ScxSOPPKLLL79ckydP1h/+8AetWrVKb775ZtCuroZcsJCkl156SZGRkbrjjjt08uRJ9e7dW0uWLAneGl6ErIULFyozM1OZmZm/CV5ubhSsW2+9VQcPHtS4ceOUk5OjLl26aP78+b/5QiekGTNmSJL69etX4fV33333grUccD49e/bU3LlzlZaWpkmTJikpKUnTpk3TqFGjgnJ8bpsOAAD8hnIYAAD4DcECAAD4DcECAAD4DcECAAD4DcECAAD4DcECAAD4DcECAAD4DcECAAD4DcECAAD4DcECAAD4DcECAAD4DcECAAD4zf8DV44VgxRqNQsAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "\n",
    "def relu_funcrion(x):\n",
    "    return np.maximum(0, x)\n",
    "\n",
    "\n",
    "x = np.arange(-6, 6, 0.1)\n",
    "y = relu_funcrion(x)\n",
    "\n",
    "plt.plot(x, y)\n",
    "plt.show()\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.6Tanh激活函数\n",
    "\n",
    "公式如下：\n",
    "\n",
    "![image-20230408195659833](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/image-20230408195659833.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjgAAAGdCAYAAAAfTAk2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAABCLklEQVR4nO3deXxU9b3/8ffMJJkkkJVsBAKERRZZAgnEqK1acglKW+nPWmlRJFW4WlBpsGh6FVSs1KVeqnKlLii2Uq2t1rWpGIpWG1mCqCiiKBAgTAKEzGSBLDPn90fIaGRLIJMzM3k9H4/zSObM95zzOfOA5J3v+Z7vsRiGYQgAACCIWM0uAAAAoLMRcAAAQNAh4AAAgKBDwAEAAEGHgAMAAIIOAQcAAAQdAg4AAAg6BBwAABB0QswuwAwej0fl5eWKioqSxWIxuxwAANAOhmGopqZGqampslpP3kfTLQNOeXm50tLSzC4DAACcht27d6tv374nbdMtA05UVJSklg8oOjra5GoAAEB7uFwupaWleX+Pn0y3DDitl6Wio6MJOAAABJj2DC9hkDEAAAg6BBwAABB0CDgAACDoEHAAAEDQIeAAAICgQ8ABAABBh4ADAACCDgEHAAAEHQIOAAAIOj4NOO+8845+8IMfKDU1VRaLRX//+99Puc3atWs1btw42e12DR48WE8//fQxbZYtW6YBAwYoPDxc2dnZWr9+fecXDwAAApZPA05dXZ3GjBmjZcuWtav9jh07NGXKFF100UXavHmz5s2bp2uvvVb//Oc/vW2ef/55FRQUaNGiRdq0aZPGjBmjvLw8VVZW+uo0AABAgLEYhmF0yYEsFr300kuaOnXqCdvccsstev3117VlyxbvumnTpqm6ulpFRUWSpOzsbI0fP16PPPKIJMnj8SgtLU033HCDbr311nbV4nK5FBMTI6fTybOoAAAIEB35/e1XD9ssKSlRbm5um3V5eXmaN2+eJKmxsVGlpaUqLCz0vm+1WpWbm6uSkpIT7rehoUENDQ3e1y6Xq3MLBwB0OcMw1Owx1NjsUWOzR01ujxrdHjW5DTW5W143uw01e1rWuT0t7d0ej9weff3VMOTxtLzvMVoWt0fyGIYMw5DH0NH1Lcc0jr42JO/3rfW0tJEMGUe/Hm0kedu3bOldLcN7Pq2v277xzV6Ib/dJfLuL4lQ9Fifr0jBOuXXHZPaP0/dHp3bqPjvCrwKOw+FQcnJym3XJyclyuVw6fPiwDh06JLfbfdw2n3322Qn3u2TJEt15550+qRkA0H4ejyHn4SYdqm9U9eEmVdc3ynm4STVHmlVzpFmuI02qa2hWfYNbtQ3Nqm90q76xWYebPDrS5NbhRrcamt060uRRQ7Nbni65BoHT0dDsIeD4WmFhoQoKCryvXS6X0tLSTKwIAIJPfWOz9hw6rN1V9Sp3HlGF84j2OY+osuaI9tc06GBdo6rqGuX2USqxWqRQm1VhNqtCQ6wKsVoUarMqxGaRzWpRiNWiEOvXr20Wi6xHv9qsrd9L1qPrra3fWyySRS3tLS1DLixq+dryWrLI0vLVIqn1e7Vs3/q9jm7T8vXoa337tdq+bm3/zRNt8+LrfZyI5Zj2p+/b+zqZMX1jz+BIZ86vAk5KSooqKirarKuoqFB0dLQiIiJks9lks9mO2yYlJeWE+7Xb7bLb7T6pGQC6E8MwtLf6sLY5avTl/lptr6zVl/vrtOtgnQ7UNrZ7P1H2EMVEhio2MlQxEaGKsocqKjxEUeGh6hkeop52myLDQtTDblNEaIgiw2yKCLMpItSm8FCr7CE22UOsCmtdbFaF2Jj5BF/zq4CTk5OjN954o8261atXKycnR5IUFhamzMxMFRcXewcrezweFRcXa+7cuV1dLgAEvf01DSrdVaVNZdX6pNypT8pdqq5vOmH76PAQpcVHKjU2QinR4UqJCVdydLgSo+xK6BmmhJ52xfcIUyhhBD7m04BTW1ur7du3e1/v2LFDmzdvVnx8vPr166fCwkLt3btXzzzzjCTpuuuu0yOPPKIFCxbo5z//udasWaO//OUvev311737KCgo0NVXX62srCxNmDBBS5cuVV1dnfLz8315KgDQLRyqa9Q7X+zXu18c0IadVdp5sP6YNiFWiwYn9dTgpJ4alNjyNT2hh9LiIhUTGWpC1cCxfBpwNm7cqIsuusj7unUczNVXX62nn35a+/btU1lZmff99PR0vf766/rlL3+p3//+9+rbt6+eeOIJ5eXledtcccUV2r9/vxYuXCiHw6GMjAwVFRUdM/AYANA+X+2v1esf7dOabZX6cHd1m4G7Fos0NDlKmf3jNKpPjEb2idGQ5J6yh9jMKxhohy6bB8efMA8OgO5un/OwXtlcrlc/KteWvW2nzhiaHKULhiYqZ2AvjesXR68M/EbAzoMDAPAdwzD03vaDeqZkp97aWuHtqbFZLfrOkATlnZ2iC85KVGpshLmFAp2AgAMAQa6x2aMXSnfryXd36Kv9dd71EwbE69Kxqbp4ZG/F9wgzsUKg8xFwACBINTZ79NfSPVr2r+3aW31YktTTHqL/N66Prjqnv4YkR5lcIeA7BBwACDKGYejVj/bpvqLPtOdQS7BJirLrugsG6Sfj09TTzo9+BD/+lQNAEPlyf60WvrxF720/KElK6GnXLy4cpJ9l91N4KHc+ofsg4ABAEDjS5NYja7brD+98qSa3IXuIVb+4cLBmf3egIsIINuh+CDgAEOC+3F+ruas+0NZ9Lbd7XzQ0UXf+cKT69Yo0uTLAPAQcAAhQhmHor6V7tPDlT3S4ya1ePcL0mx+NVN7ZKd6HNALdFQEHAALQ4Ua3Cl/8SH/fXC5JOm9wL/3vTzKUFB1ucmWAfyDgAECAOVjboGuf2agPyqpls1pU8F9n6boLBslmpdcGaEXAAYAAsvNAnWY+tV47D9YrJiJUj12VqeyBvcwuC/A7BBwACBCbyg7p2pUbVVXXqL5xEXo6f4IGJ/U0uyzALxFwACAAlO6q0lVPrld9o1uj+sToyZlZSopivA1wIgQcAPBzW/Y6NXPFBtU3unX+4AT94apM9WA2YuCk+B8CAH7s84oaXfXkOtU0NGvCgHg9PiOLifuAdrCaXQAA4Ph2HKjT9CfW6VB9k8b0bbksRbgB2oeAAwB+yFnfpJlPrdf+mgYNS4nSyp9PUFR4qNllAQGDgAMAfsbtMXTDcx9o18F69YmN0B+vyVZsZJjZZQEBhYADAH7m/n9u0zuf71d4qFWPzchUYpTd7JKAgEPAAQA/8tpH5Vr+9peSpPt+PEZnp8aYXBEQmAg4AOAntu5z6VcvfCRJ+u/vDtQPx6SaXBEQuAg4AOAHGprduum5D3S4ya3vDEnQgsnDzC4JCGgEHADwAw+u/lyfV9QqoWeYll6RwYMzgTNEwAEAk5XuqtJj73wlSfrNj0apV08GFQNnioADACaqb2zW/L98KMOQ/t+4Pso7O8XskoCgQMABABPd+4/PtPNgvXrHhGvRD842uxwgaBBwAMAk7391UCtLdkmS7r1stGIimKkY6CwEHAAwQbPbozte+USS9NMJ/fTdsxJNrggILgQcADDBnzfs1meOGsVEhGpB3lCzywGCDgEHALpYdX2jHnxzmySp4L/OUlwPnjMFdDYCDgB0saVvfaFD9U06K7mnpmf3M7scICh1ScBZtmyZBgwYoPDwcGVnZ2v9+vUnbHvhhRfKYrEcs0yZMsXbZubMmce8P3ny5K44FQA4I59X1OiP77cMLF70g7MVYuPvTMAXQnx9gOeff14FBQVavny5srOztXTpUuXl5Wnbtm1KSko6pv2LL76oxsZG7+uDBw9qzJgxuvzyy9u0mzx5sp566inva7udibEA+DfDMLT4tU/l9hiaNCJZ5w1OMLskIGj5/E+HBx98ULNmzVJ+fr5GjBih5cuXKzIyUitWrDhu+/j4eKWkpHiX1atXKzIy8piAY7fb27SLi4vz9akAwBl5d/sB/fuLAwqzWXXblBFmlwMENZ8GnMbGRpWWlio3N/frA1qtys3NVUlJSbv28eSTT2ratGnq0aNHm/Vr165VUlKShg4dquuvv14HDx484T4aGhrkcrnaLADQlQzD0O/f+kKSNP2cfurXK9LkioDg5tOAc+DAAbndbiUnJ7dZn5ycLIfDccrt169fry1btujaa69ts37y5Ml65plnVFxcrHvvvVdvv/22Lr74Yrnd7uPuZ8mSJYqJifEuaWlpp39SAHAa3tt+UBt3HVJYiFXXXTDI7HKAoOfzMThn4sknn9SoUaM0YcKENuunTZvm/X7UqFEaPXq0Bg0apLVr12rixInH7KewsFAFBQXe1y6Xi5ADoMsYhqHfF38uSfrZhH5Kjg43uSIg+Pm0BychIUE2m00VFRVt1ldUVCgl5eQPlKurq9Nzzz2na6655pTHGThwoBISErR9+/bjvm+32xUdHd1mAYCu8p8vD2rDzpbem+svpPcG6Ao+DThhYWHKzMxUcXGxd53H41FxcbFycnJOuu0LL7yghoYGXXnllac8zp49e3Tw4EH17t37jGsGgM70zbE39N4AXcfnd1EVFBTo8ccf18qVK7V161Zdf/31qqurU35+viRpxowZKiwsPGa7J598UlOnTlWvXr3arK+trdWvfvUrvf/++9q5c6eKi4t16aWXavDgwcrLy/P16QBAh5R8eVDrd1YpzMbYG6Ar+XwMzhVXXKH9+/dr4cKFcjgcysjIUFFRkXfgcVlZmazWtjlr27Ztevfdd/Xmm28esz+bzaaPPvpIK1euVHV1tVJTUzVp0iQtXryYuXAA+J3fF7f03vx0QppSYui9AbqKxTAMw+wiuprL5VJMTIycTifjcQD4zEd7qvXDR95TqM2idxZcpN4xEWaXBAS0jvz+Zo5wAPCRp9/bKUmaMqo34QboYgQcAPCBypojevWjcklS/nnpJlcDdD8EHADwgT+v260mt6Gx/WI1Ji3W7HKAboeAAwCdrLHZoz+ta3liOL03gDkIOADQyd74eJ/21zQoOdqui0eefFJTAL5BwAGATmQYhp56b4ck6apz+ivUxo9ZwAz8zwOATvTB7mp9uMepsBCrfjqhn9nlAN0WAQcAOtHK/+yUJF06JlW9ejL5KGAWAg4AdBJnfZP+scUhSboqp7/J1QDdGwEHADrJKx+Vq7HZo2EpURrVJ8bscoBujYADAJ3krxt3S5J+nNlXFovF5GqA7o2AAwCd4POKGn24x6kQq0U/GtvH7HKAbo+AAwCd4K+leyRJ3xuWxOBiwA8QcADgDDW5PXpx015JLZenAJiPgAMAZ+jtbft1oLZBCT3DdNGwJLPLASACDgCcsdbLU1Mz+jBzMeAn+J8IAGegqq5RxZ9VSJJ+nMXlKcBfEHAA4Ay8vHmvmtyGRvWJ0bCUaLPLAXAUAQcAzsBrH+2TJP2/cdwaDvgTAg4AnKby6sMq3XVIFot0yajeZpcD4BsIOABwmt74uKX3ZvyAeCVHh5tcDYBvIuAAwGl6/WjAmULvDeB3CDgAcBr2Vh/WB2XVsliki0emmF0OgG8h4ADAafjH0d6bCQPilcTlKcDvEHAA4DS03j31/dFcngL8EQEHADpod1W9Nu+ultUi5XF5CvBLBBwA6KB/bDl6eSo9XklRXJ4C/BEBBwA66PWPHZKkKaNTTa4EwIkQcACgA3ZX1evDo5enJp/N5SnAXxFwAKAD3vy05cGaE9LjlRhlN7kaACdCwAGADije2hJw/msEvTeAP+uSgLNs2TINGDBA4eHhys7O1vr160/Y9umnn5bFYmmzhIe3HcRnGIYWLlyo3r17KyIiQrm5ufriiy98fRoAujnXkSat31ElScodnmRyNQBOxucB5/nnn1dBQYEWLVqkTZs2acyYMcrLy1NlZeUJt4mOjta+ffu8y65du9q8f9999+mhhx7S8uXLtW7dOvXo0UN5eXk6cuSIr08HQDf29rb9avYYGpzUU/179TC7HAAn4fOA8+CDD2rWrFnKz8/XiBEjtHz5ckVGRmrFihUn3MZisSglJcW7JCcne98zDENLly7VbbfdpksvvVSjR4/WM888o/Lycv3973/39ekA6MZaL09NpPcG8Hs+DTiNjY0qLS1Vbm7u1we0WpWbm6uSkpITbldbW6v+/fsrLS1Nl156qT755BPvezt27JDD4Wizz5iYGGVnZ59wnw0NDXK5XG0WAOiIZrdH/9q2X5KUOzz5FK0BmM2nAefAgQNyu91temAkKTk5WQ6H47jbDB06VCtWrNDLL7+sP/3pT/J4PDr33HO1Z88eSfJu15F9LlmyRDExMd4lLS3tTE8NQDezcdchOQ83KS4yVOP6xZldDoBT8Lu7qHJycjRjxgxlZGToggsu0IsvvqjExET94Q9/OO19FhYWyul0epfdu3d3YsUAuoPWy1MXDU2SzWoxuRoAp+LTgJOQkCCbzaaKioo26ysqKpSS0r5bLENDQzV27Fht375dkrzbdWSfdrtd0dHRbRYA6IjirS03Rkzk8hQQEHwacMLCwpSZmani4mLvOo/Ho+LiYuXk5LRrH263Wx9//LF69255Ym96erpSUlLa7NPlcmndunXt3icAdMRX+2v11YE6hdos+u5ZCWaXA6AdQnx9gIKCAl199dXKysrShAkTtHTpUtXV1Sk/P1+SNGPGDPXp00dLliyRJN11110655xzNHjwYFVXV+v+++/Xrl27dO2110pqucNq3rx5uvvuuzVkyBClp6fr9ttvV2pqqqZOnerr0wHQDbX23mSn91JUeKjJ1QBoD58HnCuuuEL79+/XwoUL5XA4lJGRoaKiIu8g4bKyMlmtX3ckHTp0SLNmzZLD4VBcXJwyMzP1n//8RyNGjPC2WbBggerq6jR79mxVV1fr/PPPV1FR0TETAgJAZ3jr6PgbJvcDAofFMAzD7CK6msvlUkxMjJxOJ+NxAJyU83CTxi1eLbfH0L8XXKS0+EizSwK6rY78/va7u6gAwJ/8Z/sBuY/OXky4AQIHAQcATuKdLw5Ikr47JNHkSgB0BAEHAE7AMAy983nL7MXf4e4pIKAQcADgBHYcqNPe6sMKs1mVnR5vdjkAOoCAAwAn8O+jl6eyBsQpMsznN50C6EQEHAA4gdbLU989i/E3QKAh4ADAcTQ2e1Ty1UFJ0neGMP4GCDQEHAA4jk1lh1Tf6FZCzzANT2G+LCDQEHAA4DhaL0+dPzhBVp4eDgQcAg4AHEfrAGPG3wCBiYADAN9ysLZBW8qdklp6cAAEHgIOAHzLu9sPyDCkYSlRSormIb5AICLgAMC3cHkKCHwEHAD4BsMw9O7RgMPt4UDgIuAAwDfsOFAnh+uIwmxWjR/A4xmAQEXAAYBvaJ3cb2y/WIWH2kyuBsDpIuAAwDeUfNkScHIG9TK5EgBngoADAEcZhqH3j/bg5Awk4ACBjIADAEd9UVmrA7WNsodYldEv1uxyAJwBAg4AHNV6eSprQJzsIYy/AQIZAQcAjvKOv+HyFBDwCDgAIMnjMfT+DgYYA8GCgAMAkj5z1Ki6vkmRYTaN7htrdjkAzhABBwD09fw34wfEK9TGj0Yg0PG/GADE/DdAsCHgAOj23B5D63YwwBgIJgQcAN3eJ+VO1RxpVpQ9RGenRptdDoBOQMAB0O21Xp6akB6vEMbfAEGB/8kAuj3v4xkYfwMEDQIOgG7N7TG0cdchSVJ2OgEHCBYEHADd2mcOl2qONKunPUTDe0eZXQ6ATtIlAWfZsmUaMGCAwsPDlZ2drfXr15+w7eOPP67vfOc7iouLU1xcnHJzc49pP3PmTFksljbL5MmTfX0aAILQhh1VkqRx/eMYfwMEEZ//b37++edVUFCgRYsWadOmTRozZozy8vJUWVl53PZr167VT3/6U/3rX/9SSUmJ0tLSNGnSJO3du7dNu8mTJ2vfvn3e5c9//rOvTwVAEFq/syXgTBgQZ3IlADqTzwPOgw8+qFmzZik/P18jRozQ8uXLFRkZqRUrVhy3/bPPPqtf/OIXysjI0LBhw/TEE0/I4/GouLi4TTu73a6UlBTvEhfHDycAHWMYhtbvaBl/M4HxN0BQ8WnAaWxsVGlpqXJzc78+oNWq3NxclZSUtGsf9fX1ampqUnx8fJv1a9euVVJSkoYOHarrr79eBw8ePOE+Ghoa5HK52iwAsONAnQ7UNijMZtXovjFmlwOgE/k04Bw4cEBut1vJyclt1icnJ8vhcLRrH7fccotSU1PbhKTJkyfrmWeeUXFxse699169/fbbuvjii+V2u4+7jyVLligmJsa7pKWlnf5JAQgaG45enspIi1V4qM3kagB0phCzCziZ3/72t3ruuee0du1ahYeHe9dPmzbN+/2oUaM0evRoDRo0SGvXrtXEiROP2U9hYaEKCgq8r10uFyEHgNYdHWA8Pp1L3ECw8WkPTkJCgmw2myoqKtqsr6ioUEpKykm3feCBB/Tb3/5Wb775pkaPHn3StgMHDlRCQoK2b99+3Pftdruio6PbLADQ2oPD+Bsg+Pg04ISFhSkzM7PNAOHWAcM5OTkn3O6+++7T4sWLVVRUpKysrFMeZ8+ePTp48KB69+7dKXUDCH77nIe1u+qwrBZpXL9Ys8sB0Ml8fhdVQUGBHn/8ca1cuVJbt27V9ddfr7q6OuXn50uSZsyYocLCQm/7e++9V7fffrtWrFihAQMGyOFwyOFwqLa2VpJUW1urX/3qV3r//fe1c+dOFRcX69JLL9XgwYOVl5fn69MBECTWH708NSI1WlHhoSZXA6Cz+XwMzhVXXKH9+/dr4cKFcjgcysjIUFFRkXfgcVlZmazWr3PWo48+qsbGRv34xz9us59FixbpjjvukM1m00cffaSVK1equrpaqampmjRpkhYvXiy73e7r0wEQJFoDzoQBXJ4CgpHFMAzD7CK6msvlUkxMjJxOJ+NxgG5q0v++rc8rarX8ynGaPJLL20Ag6Mjvb+YlB9DtHKpr1OcVLZe9xw+IP0VrAIGIgAOg22m9e2pQYg/16smlbSAYEXAAdDtf3x5O7w0QrAg4ALqdjbtanj+V1Z+AAwQrAg6AbuVIk1tb9jolSVk8QRwIWgQcAN3KR3ucanIbSuhpV7/4SLPLAeAjBBwA3crGXS3jb7L6x8lisZhcDQBfIeAA6FZKdx4df8PlKSCoEXAAdBsej6HSspaAk9mfgAMEMwIOgG7jqwO1qq5vkj3EqrNTY8wuB4APEXAAdBsbj16eykiLVVgIP/6AYMb/cADdhnf+G8bfAEGPgAOg29i4s/UOKib4A4IdAQdAt7C/pkE7D9ZLksb1owcHCHYEHADdQunRy1NnJfdUTGSoydUA8DUCDoBuofToBH+ZXJ4CugUCDoBu4esHbHJ5CugOCDgAgh4P2AS6HwIOgKD38V4esAl0NwQcAEGv9BuXp3jAJtA9EHAABL3WgMPzp4Dug4ADIKgZhqFNRwPOOAIO0G0QcAAEtV0H63WwrlFhNqtG9ok2uxwAXYSAAyCotV6eGtU3RvYQm8nVAOgqBBwAQa20jPE3QHdEwAEQ1Lzjb3j+FNCtEHAABC3XkSZtq6iRJI3rH2tuMQC6FAEHQNDaXFYtw5D6xUcqKSrc7HIAdCECDoCgxfw3QPdFwAEQtDaVMf8N0F0RcAAEJbfH0Adl1ZKkTAYYA91OlwScZcuWacCAAQoPD1d2drbWr19/0vYvvPCChg0bpvDwcI0aNUpvvPFGm/cNw9DChQvVu3dvRUREKDc3V1988YUvTwFAgPm8oka1Dc3qEWbT0JQos8sB0MV8HnCef/55FRQUaNGiRdq0aZPGjBmjvLw8VVZWHrf9f/7zH/30pz/VNddcow8++EBTp07V1KlTtWXLFm+b++67Tw899JCWL1+udevWqUePHsrLy9ORI0d8fToAAkTr5amx/eJks/KATaC7sRiGYfjyANnZ2Ro/frweeeQRSZLH41FaWppuuOEG3Xrrrce0v+KKK1RXV6fXXnvNu+6cc85RRkaGli9fLsMwlJqaqvnz5+vmm2+WJDmdTiUnJ+vpp5/WtGnTTlmTy+VSTEyMnE6noqOZuh0IRgV/2awXN+3VjROHqOC/zjK7HACdoCO/v33ag9PY2KjS0lLl5uZ+fUCrVbm5uSopKTnuNiUlJW3aS1JeXp63/Y4dO+RwONq0iYmJUXZ29gn32dDQIJfL1WYBENw2cQcV0K35NOAcOHBAbrdbycnJbdYnJyfL4XAcdxuHw3HS9q1fO7LPJUuWKCYmxrukpaWd1vkACAwHahu082C9JCkjLdbcYgCYolvcRVVYWCin0+lddu/ebXZJAHyotffmrOSeiokINbkaAGbwacBJSEiQzWZTRUVFm/UVFRVKSUk57jYpKSknbd/6tSP7tNvtio6ObrMACF48YBOATwNOWFiYMjMzVVxc7F3n8XhUXFysnJyc426Tk5PTpr0krV692ts+PT1dKSkpbdq4XC6tW7fuhPsE0L3wgE0AIb4+QEFBga6++mplZWVpwoQJWrp0qerq6pSfny9JmjFjhvr06aMlS5ZIkm666SZdcMEF+t3vfqcpU6boueee08aNG/XYY49JkiwWi+bNm6e7775bQ4YMUXp6um6//XalpqZq6tSpvj4dAH6usdmjD/c4JdGDA3RnPg84V1xxhfbv36+FCxfK4XAoIyNDRUVF3kHCZWVlslq/7kg699xztWrVKt1222369a9/rSFDhujvf/+7Ro4c6W2zYMEC1dXVafbs2aqurtb555+voqIihYfzMD2gu/uk3KnGZo/iIkOVntDD7HIAmMTn8+D4I+bBAYLXE//+Sne/vlW5w5P0xNXjzS4HQCfym3lwAKCr8YBNABIBB0AQMQxDpa0T/DHAGOjWCDgAgsbe6sOqcDUoxGrR6L6xZpcDwEQEHABBo7X35uzUaEWE2UyuBoCZCDgAgoZ3/hvG3wDdHgEHQNDYVFYtiflvABBwAASJ+sZmfbrPJYkZjAEQcAAEiQ93O+X2GOodE67U2AizywFgMgIOgKDA/DcAvomAAyAoMP8NgG8i4AAIeB7PNyb4owcHgAg4AILAl/tr5TzcpIhQm0ak8nw5AAQcAEFg49HemzFpMQq18WMNAAEHQBDYsLNKkpTVP97kSgD4CwIOgIDnHX8zgPE3AFoQcAAEtP01Ddp1sF4WCxP8AfgaAQdAQCvd1XJ5amhylGIiQk2uBoC/IOAACGgbd3J7OIBjEXAABLTWO6iyGH8D4BsIOAAC1pEmtz4pd0riDioAbRFwAASsD3dXq8ltKCnKrr5xPGATwNcIOAAC1jcvT1ksFpOrAeBPCDgAAtbGoxP8ZXJ5CsC3EHAABKRvPmBzPAOMAXwLAQdAQNq+v1auI82KCLVpeG8esAmgLQIOgIDUOv9NRlosD9gEcAx+KgAISBuPzmDM/DcAjoeAAyAgtT5BfPwABhgDOBYBB0DA2ec8rN1Vh2W1SON4RAOA4yDgAAg463e09N6cnRqjnvYQk6sB4I8IOAACTmvAmZDO5SkAx+fTgFNVVaXp06crOjpasbGxuuaaa1RbW3vS9jfccIOGDh2qiIgI9evXTzfeeKOcTmebdhaL5Zjlueee8+WpAPAjjL8BcCo+7dudPn269u3bp9WrV6upqUn5+fmaPXu2Vq1addz25eXlKi8v1wMPPKARI0Zo165duu6661ReXq6//vWvbdo+9dRTmjx5svd1bGysL08FgJ84VNeozyta/lBigj8AJ+KzgLN161YVFRVpw4YNysrKkiQ9/PDDuuSSS/TAAw8oNTX1mG1Gjhypv/3tb97XgwYN0m9+8xtdeeWVam5uVkjI1+XGxsYqJSXFV+UD8FOtvTeDk3qqV0+7ydUA8Fc+u0RVUlKi2NhYb7iRpNzcXFmtVq1bt67d+3E6nYqOjm4TbiRpzpw5SkhI0IQJE7RixQoZhnHCfTQ0NMjlcrVZAAQmxt8AaA+f9eA4HA4lJSW1PVhIiOLj4+VwONq1jwMHDmjx4sWaPXt2m/V33XWXvve97ykyMlJvvvmmfvGLX6i2tlY33njjcfezZMkS3Xnnnad3IgD8SmsPzgTG3wA4iQ734Nx6663HHeT7zeWzzz4748JcLpemTJmiESNG6I477mjz3u23367zzjtPY8eO1S233KIFCxbo/vvvP+G+CgsL5XQ6vcvu3bvPuD4AXa+uoVlbylt6YMfTgwPgJDrcgzN//nzNnDnzpG0GDhyolJQUVVZWtlnf3NysqqqqU46dqamp0eTJkxUVFaWXXnpJoaGhJ22fnZ2txYsXq6GhQXb7sdfk7Xb7cdcDCCybyg7J7THUJzZCfWIjzC4HgB/rcMBJTExUYmLiKdvl5OSourpapaWlyszMlCStWbNGHo9H2dnZJ9zO5XIpLy9Pdrtdr7zyisLDw095rM2bNysuLo4QAwS5DYy/AdBOPhuDM3z4cE2ePFmzZs3S8uXL1dTUpLlz52ratGneO6j27t2riRMn6plnntGECRPkcrk0adIk1dfX609/+lObAcGJiYmy2Wx69dVXVVFRoXPOOUfh4eFavXq17rnnHt18882+OhUAfmIdAQdAO/l0Hpxnn31Wc+fO1cSJE2W1WnXZZZfpoYce8r7f1NSkbdu2qb6+XpK0adMm7x1WgwcPbrOvHTt2aMCAAQoNDdWyZcv0y1/+UoZhaPDgwXrwwQc1a9YsX54KAJM1NLu1eXe1JCb4A3BqFuNk91cHKZfLpZiYGO8t6AD838adVfrx8hL16hGmjbflymKxmF0SgC7Wkd/fPIsKQEBovTyVNSCOcAPglAg4AAJCyZcHJUk5A3uZXAmAQEDAAeD3Gprd2rirpQcnZ1CCydUACAQEHAB+78PdTh1p8qhXjzCdldzT7HIABAACDgC/13p56pxBvRh/A6BdCDgA/F7JVwckMf4GQPsRcAD4tSNNbm0qq5Yk5Qwi4ABoHwIOAL+2adchNTZ7lBRl18CEHmaXAyBAEHAA+LWSr47eHs74GwAdQMAB4NeY/wbA6SDgAPBb9Y3N+nBPtSTpXOa/AdABBBwAfmvjzkNqchvqExuhtPgIs8sBEEAIOAD8Vuv4m3MGMv4GQMcQcAD4Le/4G24PB9BBBBwAfqnmSJM+3uuURMAB0HEEHAB+ad1XVXJ7DPWLj1SfWMbfAOgYAg4Av/TvL/ZLkr4zhLunAHQcAQeAX/r3Fy3Pn/ruWYkmVwIgEBFwAPid3VX1+upAnWxWC+NvAJwWAg4Av9PaezM2LVbR4aEmVwMgEBFwAPid1vE3XJ4CcLoIOAD8SrPbo3e3t/TgMMAYwOki4ADwKx/ucarmSLOiw0M0um+s2eUACFAEHAB+pfXy1PlDEmSz8ngGAKeHgAPAr3hvDx/C+BsAp4+AA8BvOA83afPuakktPTgAcLoIOAD8RsmXB+T2GBqY2EN94yLNLgdAACPgAPAb73B5CkAnIeAA8AuGYeidz1vnv+HyFIAzQ8AB4Be2V9Zqz6HDCrNZlZ3O4xkAnBkCDgC/8NbWSklSzqBe6mEPMbkaAIHOpwGnqqpK06dPV3R0tGJjY3XNNdeotrb2pNtceOGFslgsbZbrrruuTZuysjJNmTJFkZGRSkpK0q9+9Ss1Nzf78lQA+Fjx1gpJUu7wJJMrARAMfPpn0vTp07Vv3z6tXr1aTU1Nys/P1+zZs7Vq1aqTbjdr1izddddd3teRkV/fTeF2uzVlyhSlpKToP//5j/bt26cZM2YoNDRU99xzj8/OBYDvVNU1alPZIUnS94Ynm1wNgGDgs4CzdetWFRUVacOGDcrKypIkPfzww7rkkkv0wAMPKDU19YTbRkZGKiUl5bjvvfnmm/r000/11ltvKTk5WRkZGVq8eLFuueUW3XHHHQoLC/PJ+QDwnX99VimPIQ3vHa0+sRFmlwMgCPjsElVJSYliY2O94UaScnNzZbVatW7dupNu++yzzyohIUEjR45UYWGh6uvr2+x31KhRSk7++q+8vLw8uVwuffLJJ8fdX0NDg1wuV5sFgP8o/ozLUwA6l896cBwOh5KS2v6wCgkJUXx8vBwOxwm3+9nPfqb+/fsrNTVVH330kW655RZt27ZNL774one/3ww3kryvT7TfJUuW6M477zyT0wHgI43NHr3zecv8NxO5PAWgk3Q44Nx666269957T9pm69atp13Q7Nmzvd+PGjVKvXv31sSJE/Xll19q0KBBp7XPwsJCFRQUeF+7XC6lpaWddo0AOs+6HQdV29CsxCi7RveJMbscAEGiwwFn/vz5mjlz5knbDBw4UCkpKaqsrGyzvrm5WVVVVSccX3M82dnZkqTt27dr0KBBSklJ0fr169u0qaho6d4+0X7tdrvsdnu7jwmg6xQfvT38e0OTZOXp4QA6SYcDTmJiohITTz2Nek5Ojqqrq1VaWqrMzExJ0po1a+TxeLyhpT02b94sSerdu7d3v7/5zW9UWVnpvQS2evVqRUdHa8SIER08GwBmMgxDbx29PXwi428AdCKfDTIePny4Jk+erFmzZmn9+vV67733NHfuXE2bNs17B9XevXs1bNgwb4/Ml19+qcWLF6u0tFQ7d+7UK6+8ohkzZui73/2uRo8eLUmaNGmSRowYoauuukoffvih/vnPf+q2227TnDlz6KUBAsznFUdnLw6x8vRwAJ3KpxP9Pfvssxo2bJgmTpyoSy65ROeff74ee+wx7/tNTU3atm2b9y6psLAwvfXWW5o0aZKGDRum+fPn67LLLtOrr77q3cZms+m1116TzWZTTk6OrrzySs2YMaPNvDkAAkNr7815g3opMozZiwF0HothGIbZRXQ1l8ulmJgYOZ1ORUdHm10O0G396P/e0wdl1bp76khdeU5/s8sB4Oc68vubZ1EBMMXe6sP6oKxaFos0aQS3hwPoXAQcAKb4x8f7JEnjB8QrKTrc5GoABBsCDgBTvPZRS8D5/ujeJlcCIBgRcAB0uT2H6rV5d8vlqckj2z8vFgC0FwEHQJf7x8ctj1XJTo9XUhSXpwB0PgIOgC732tHxN1NGcXkKgG8QcAB0qd1V9fpwd7WsFimPy1MAfISAA6BL/WNLS+9NdnovLk8B8BkCDoAu9frRu6emcPcUAB8i4ADoMrur6vXhHqes3D0FwMcIOAC6zOtHBxefM7CXEnrycFwAvkPAAdAlDMPQ30r3SJJ+MCbV5GoABDsCDoAu8eEep76orFV4qJXxNwB8joADoEu8sHG3JGny2SmKDg81uRoAwY6AA8DnjjS59cqH5ZKky7PSTK4GQHdAwAHgc29+WqGaI83qExuhnIG9zC4HQDdAwAHgc62Xpy4b10dWq8XkagB0BwQcAD5VXn1Y724/IEn6cSaXpwB0DQIOAJ96cdMeGUbLk8P79Yo0uxwA3QQBB4DPGIahvx6d++bHmX1NrgZAd0LAAeAzG3Ye0s6D9YoMs+mSUcx9A6DrEHAA+MzKkp2SpB+MTlUPe4i5xQDoVgg4AHxin/OwirY4JElXnzvA3GIAdDsEHAA+8ceSXXJ7DGWnx2tEarTZ5QDoZgg4ADrdkSa3/ry+TJKUf94Ac4sB0C0RcAB0ulc2l+tQfZP6xEYod3iy2eUA6IYIOAA6lWEYWvHeDknSjJz+CrHxYwZA1+MnD4BOtW5HlT5z1Cgi1KZp4/uZXQ6AboqAA6BTPXW09+ZH4/ooJjLU5GoAdFcEHACdZtfBOq3+tEKSlM+t4QBMRMAB0GkeWbNdHkO64KxEDUmOMrscAN2YTwNOVVWVpk+frujoaMXGxuqaa65RbW3tCdvv3LlTFovluMsLL7zgbXe895977jlfngqAUyg7WK8XP9grSbopd4jJ1QDo7nw6d/r06dO1b98+rV69Wk1NTcrPz9fs2bO1atWq47ZPS0vTvn372qx77LHHdP/99+viiy9us/6pp57S5MmTva9jY2M7vX4A7ffIv76Q22Pou2claly/OLPLAdDN+SzgbN26VUVFRdqwYYOysrIkSQ8//LAuueQSPfDAA0pNTT1mG5vNppSUlDbrXnrpJf3kJz9Rz54926yPjY09pi0Ac5QdrNeLm4723kyk9waA+Xx2iaqkpESxsbHecCNJubm5slqtWrduXbv2UVpaqs2bN+uaa6455r05c+YoISFBEyZM0IoVK2QYxgn309DQIJfL1WYB0HmW/Wu7mj2GvjMkQZn96b0BYD6f9eA4HA4lJSW1PVhIiOLj4+VwONq1jyeffFLDhw/Xueee22b9XXfdpe9973uKjIzUm2++qV/84heqra3VjTfeeNz9LFmyRHfeeefpnQiAk9pdVa+/bdojSZrH2BsAfqLDPTi33nrrCQcCty6fffbZGRd2+PBhrVq16ri9N7fffrvOO+88jR07VrfccosWLFig+++//4T7KiwslNPp9C67d+8+4/oAtPi/td/svYk3uxwAkHQaPTjz58/XzJkzT9pm4MCBSklJUWVlZZv1zc3NqqqqatfYmb/+9a+qr6/XjBkzTtk2OztbixcvVkNDg+x2+zHv2+32464HcGY+r6jRXza29N4w9gaAP+lwwElMTFRiYuIp2+Xk5Ki6ulqlpaXKzMyUJK1Zs0Yej0fZ2dmn3P7JJ5/UD3/4w3Yda/PmzYqLiyPEAF3IMAwtfu1TuT2GJo1IVtYAem8A+A+fjcEZPny4Jk+erFmzZmn58uVqamrS3LlzNW3aNO8dVHv37tXEiRP1zDPPaMKECd5tt2/frnfeeUdvvPHGMft99dVXVVFRoXPOOUfh4eFavXq17rnnHt18882+OhUAx/HW1kr9+4sDCrNZ9T9ThptdDgC04dN5cJ599lnNnTtXEydOlNVq1WWXXaaHHnrI+35TU5O2bdum+vr6NtutWLFCffv21aRJk47ZZ2hoqJYtW6Zf/vKXMgxDgwcP1oMPPqhZs2b58lQAfENDs1t3v/6pJOna76Srf68eJlcEAG1ZjJPdXx2kXC6XYmJi5HQ6FR0dbXY5QMB5dO2XurfoMyVF2bXm5gvV0+7Tv5UAQFLHfn/zLCoAHVLpOqJH1nwhSbr14mGEGwB+iYADoEN+88ZW1TW6lZEWq6kZfcwuBwCOi4ADoN2Ktjj08uZyWS3SHT88W1arxeySAOC4CDgA2uVAbYP+56WPJUn/fcEgZaTFmlsQAJwEAQfAKRmGodte2qKDdY0alhLFIxkA+D0CDoBTenlzuYo+cSjEatHvfjJG9hCb2SUBwEkRcACclMN5RAtf3iKp5XEMZ6fGmFwRAJwaAQfACTU2ezRn1Sa5jjRrTN8YXX/hILNLAoB2IeAAOKE7Xv1EpbsOKSo8REunjVWIjR8ZAAIDP60AHNeqdWVata5MFov00E/HKj2BxzEACBwEHADHKN1VpUWvtIy7uXnSUF00NMnkigCgYwg4ANrYW31Y1/1pk5rchi4ZlaJfMO4GQAAi4ADwqnQd0fTH39f+mgYNTY7S/T8eI4uF2YoBBB4CDgBJUlVdo658cp12HqxX37gIPf3z8erBgzQBBCgCDgA5Dzdpxop1+ryiVsnRdq269hz1jokwuywAOG38eQZ0c9X1jfr50xu0Za9LvXqE6dlrz1G/XpFmlwUAZ4SAA3Rju6vqNfOp9fpyf52iw0P0x2uyNTipp9llAcAZI+AA3dTHe5zKf3qDDtQ2KCU6XE//fLyGpUSbXRYAdAoCDtANFW+t0A1//kD1jW4NS4nSU/njGXMDIKgQcIBupLHZowfe3KbH3vlKknT+4AT935XjFB0eanJlANC5CDhAN1F2sF43/HmTPtzjlCRddU5/3f79EQoL4WZKAMGHgAMEOY/H0F837dHiVz9VTUOzosNDdN+Px2jyyBSzSwMAnyHgAEHsM4dLt/99izbsPCRJyuofp6XTMtQ3jtvAAQQ3Ag4QhJyHm/TImi+04r2dcnsMRYTadFPuEF17frpCbFySAhD8CDhAEHEdadJT7+7UE+9+pZojzZKki0em6Pbvj1BqLHdJAeg+CDhAEDhY26Bn15XpyXd3yHm4SZJ0VnJPFV4yXBcNTTK5OgDoegQcIEAZhqHNu6v1x5Jdeu2jfWp0eyRJQ5J66qbcIbpkZG9ZrTwJHED3RMABAsyeQ/V67aN9enlzubbuc3nXj+kbo5+fn67vj06VjWADoJsj4AB+zjAMba+s1dpt+1X0iUOluw553wsLser7o3trRs4AZaTFmlckAPgZAg7ghxzOI9qws0olXx3U29v2a2/1Ye97FouUnR6vH4xJ1SUjeyuuR5iJlQKAfyLgACZzHm7Sp+UufVLu1Ja9Tm3cdUh7Dh1u0yYsxKpzBvbSRUMTdcmo3kqODjepWgAIDD4LOL/5zW/0+uuva/PmzQoLC1N1dfUptzEMQ4sWLdLjjz+u6upqnXfeeXr00Uc1ZMgQb5uqqirdcMMNevXVV2W1WnXZZZfp97//vXr27OmrUwHOmNtjqMJ1RGVV9dp5oE7bK2u1fX+ttlfWHhNmJMlqkc5OjVHWgDh9d0iizhnYSxFhNhMqB4DA5LOA09jYqMsvv1w5OTl68skn27XNfffdp4ceekgrV65Uenq6br/9duXl5enTTz9VeHjLX6zTp0/Xvn37tHr1ajU1NSk/P1+zZ8/WqlWrfHUqwAl5PIZqjjTrQF2DDtY26mBtg/bXNsjhPCKH64gcziMqrz6svdWH1eQ2TrifPrERGtknWmenxmhcvzhl9ItVTzsdrABwuiyGYZz4p24nePrppzVv3rxT9uAYhqHU1FTNnz9fN998syTJ6XQqOTlZTz/9tKZNm6atW7dqxIgR2rBhg7KysiRJRUVFuuSSS7Rnzx6lpqa2qyaXy6WYmBg5nU5FR0ef0fkhMLg9hhqbPWp0e9p8PdLkVsPRr4eb3DrS2PK1vtGt+sZm1TW4VdfQrNqGZtUcaZbrSFPL18NNOlTfKOfhJnna+T8oxGpRn7gI9YuP1OCkni1LYk8NTYlSbCTjaADgVDry+9tv/kTcsWOHHA6HcnNzvetiYmKUnZ2tkpISTZs2TSUlJYqNjfWGG0nKzc2V1WrVunXr9KMf/ei4+25oaFBDQ4P3tcvlOm67M1W6q0qvfbSvzTrfxsdvHqftgb592G/XYXyrxTffN45Zd2zb1vcMGS2v1XadvOsM73ttX7ds5/F+/fo9j2HIY7T0jrR8b8jjaVnvNgx5PIaaPYbcR99v9ny9rtl99KvHo2a3oSa3R01uT7tDyOmKsoeoV88wJfS0K6GnXcnRdqXERCglxq6U6AilxUeod0wEt28DQBfxm4DjcDgkScnJyW3WJycne99zOBxKSmo7K2tISIji4+O9bY5nyZIluvPOOzu54mNtc9Tqqfd2+vw4OHNhNqvsIVbZQ21Hv1oVGWZTRKhN4aEtX3vaQxRpt6mHPUQ9w0IUFR6iqPBQRYWHKCYiVHE9whQbEaroiFCFhzI+BgD8SYcCzq233qp77733pG22bt2qYcOGnVFRna2wsFAFBQXe1y6XS2lpaZ1+nLNTozXnokEd3s6i9v9VbzlJ01Pu5Vsbf7t969sWWY45juXbbb69L0vb7azfem2xtJxlSzvJaj161haLrBbJevSrRRZZrV+vs1gkm9Xifd9qschm/dZisSjE1tIm1GZViM2iEKtFIVarQkOsCrVaFGKzKizEqjCbVaE2yzH1AwCCS4cCzvz58zVz5syTthk4cOBpFZKSkiJJqqioUO/evb3rKyoqlJGR4W1TWVnZZrvm5mZVVVV5tz8eu90uu91+WnV1xJi0WI1hsjUAAEzXoYCTmJioxMREnxSSnp6ulJQUFRcXewONy+XSunXrdP3110uScnJyVF1drdLSUmVmZkqS1qxZI4/Ho+zsbJ/UBQAAAo/VVzsuKyvT5s2bVVZWJrfbrc2bN2vz5s2qra31thk2bJheeuklSS2XMObNm6e7775br7zyij7++GPNmDFDqampmjp1qiRp+PDhmjx5smbNmqX169frvffe09y5czVt2rR230EFAACCn88GGS9cuFArV670vh47dqwk6V//+pcuvPBCSdK2bdvkdDq9bRYsWKC6ujrNnj1b1dXVOv/881VUVOSdA0eSnn32Wc2dO1cTJ070TvT30EMP+eo0AABAAPL5PDj+iHlwAAAIPB35/e2zS1QAAABmIeAAAICgQ8ABAABBh4ADAACCDgEHAAAEHQIOAAAIOgQcAAAQdAg4AAAg6BBwAABA0PHZoxr8WevkzS6Xy+RKAABAe7X+3m7PQxi6ZcCpqamRJKWlpZlcCQAA6KiamhrFxMSctE23fBaVx+NReXm5oqKiZLFYOnXfLpdLaWlp2r17N8+5OgU+q47h8+oYPq+O4fPqGD6v9uvMz8owDNXU1Cg1NVVW68lH2XTLHhyr1aq+ffv69BjR0dH8o28nPquO4fPqGD6vjuHz6hg+r/brrM/qVD03rRhkDAAAgg4BBwAABB0CTiez2+1atGiR7Ha72aX4PT6rjuHz6hg+r47h8+oYPq/2M+uz6paDjAEAQHCjBwcAAAQdAg4AAAg6BBwAABB0CDgAACDoEHB87PXXX1d2drYiIiIUFxenqVOnml2S32toaFBGRoYsFos2b95sdjl+Z+fOnbrmmmuUnp6uiIgIDRo0SIsWLVJjY6PZpfmNZcuWacCAAQoPD1d2drbWr19vdkl+acmSJRo/fryioqKUlJSkqVOnatu2bWaXFTB++9vfymKxaN68eWaX4rf27t2rK6+8Ur169VJERIRGjRqljRs3dsmxCTg+9Le//U1XXXWV8vPz9eGHH+q9997Tz372M7PL8nsLFixQamqq2WX4rc8++0wej0d/+MMf9Mknn+h///d/tXz5cv361782uzS/8Pzzz6ugoECLFi3Spk2bNGbMGOXl5amystLs0vzO22+/rTlz5uj999/X6tWr1dTUpEmTJqmurs7s0vzehg0b9Ic//EGjR482uxS/dejQIZ133nkKDQ3VP/7xD3366af63e9+p7i4uK4pwIBPNDU1GX369DGeeOIJs0sJKG+88YYxbNgw45NPPjEkGR988IHZJQWE++67z0hPTze7DL8wYcIEY86cOd7XbrfbSE1NNZYsWWJiVYGhsrLSkGS8/fbbZpfi12pqaowhQ4YYq1evNi644ALjpptuMrskv3TLLbcY559/vmnHpwfHRzZt2qS9e/fKarVq7Nix6t27ty6++GJt2bLF7NL8VkVFhWbNmqU//vGPioyMNLucgOJ0OhUfH292GaZrbGxUaWmpcnNzveusVqtyc3NVUlJiYmWBwel0ShL/lk5hzpw5mjJlSpt/ZzjWK6+8oqysLF1++eVKSkrS2LFj9fjjj3fZ8Qk4PvLVV19Jku644w7ddttteu211xQXF6cLL7xQVVVVJlfnfwzD0MyZM3XdddcpKyvL7HICyvbt2/Xwww/rv//7v80uxXQHDhyQ2+1WcnJym/XJyclyOBwmVRUYPB6P5s2bp/POO08jR440uxy/9dxzz2nTpk1asmSJ2aX4va+++kqPPvqohgwZon/+85+6/vrrdeONN2rlypVdcnwCTgfdeuutslgsJ11ax0hI0v/8z//osssuU2Zmpp566ilZLBa98MILJp9F12nv5/Xwww+rpqZGhYWFZpdsmvZ+Vt+0d+9eTZ48WZdffrlmzZplUuUIBnPmzNGWLVv03HPPmV2K39q9e7duuukmPfvsswoPDze7HL/n8Xg0btw43XPPPRo7dqxmz56tWbNmafny5V1y/JAuOUoQmT9/vmbOnHnSNgMHDtS+ffskSSNGjPCut9vtGjhwoMrKynxZol9p7+e1Zs0alZSUHPOskqysLE2fPr3LEr+Z2vtZtSovL9dFF12kc889V4899piPqwsMCQkJstlsqqioaLO+oqJCKSkpJlXl/+bOnavXXntN77zzjvr27Wt2OX6rtLRUlZWVGjdunHed2+3WO++8o0ceeUQNDQ2y2WwmVuhfevfu3eZ3oCQNHz5cf/vb37rk+AScDkpMTFRiYuIp22VmZsput2vbtm06//zzJUlNTU3auXOn+vfv7+sy/UZ7P6+HHnpId999t/d1eXm58vLy9Pzzzys7O9uXJfqN9n5WUkvPzUUXXeTtGbRa6YyVpLCwMGVmZqq4uNg7JYPH41FxcbHmzp1rbnF+yDAM3XDDDXrppZe0du1apaenm12SX5s4caI+/vjjNuvy8/M1bNgw3XLLLYSbbznvvPOOmXbg888/77LfgQQcH4mOjtZ1112nRYsWKS0tTf3799f9998vSbr88stNrs7/9OvXr83rnj17SpIGDRrEX5TfsnfvXl144YXq37+/HnjgAe3fv9/7Hr0UUkFBga6++mplZWVpwoQJWrp0qerq6pSfn292aX5nzpw5WrVqlV5++WVFRUV5xynFxMQoIiLC5Or8T1RU1DHjk3r06KFevXoxbuk4fvnLX+rcc8/VPffco5/85Cdav369HnvssS7rcSbg+ND999+vkJAQXXXVVTp8+LCys7O1Zs2arpsDAEFp9erV2r59u7Zv335M+DMMw6Sq/McVV1yh/fv3a+HChXI4HMrIyFBRUdExA48hPfroo5KkCy+8sM36p5566pSXS4FTGT9+vF566SUVFhbqrrvuUnp6upYuXarp06d3yfEtBj8RAQBAkOHCPQAACDoEHAAAEHQIOAAAIOgQcAAAQNAh4AAAgKBDwAEAAEGHgAMAAIIOAQcAAAQdAg4AAAg6BBwAABB0CDgAACDoEHAAAEDQ+f+vu9mgsrmUMQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "\n",
    "def tanh(x):\n",
    "    return (np.exp(x)-np.exp(-x))/(np.exp(x)+np.exp(-x))\n",
    "\n",
    "\n",
    "x = np.arange(-6, 6, 0.1)\n",
    "y = tanh(x)\n",
    "\n",
    "plt.plot(x, y)\n",
    "plt.show()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 三、多维数组的运算\n",
    "\n",
    "## 3.1多维数组的生成\n",
    "\n",
    "生成一维数组："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n",
      "(4,)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "A = np.array([1, 2, 3, 4])\n",
    "\n",
    "print(np.ndim(A))  # 数组的维数\n",
    "print(np.shape(A))  # 数组的形状（返回值为元组）\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "生成二维数组："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2\n",
      "(3, 2)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "B = np.array([[1, 2], [3, 4], [5, 6]])\n",
    "\n",
    "print(np.ndim(B))\n",
    "print(np.shape(B))\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.2矩阵乘积\n",
    "\n",
    "![image-20230312091729852](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/image-20230312091729852.png)\n",
    "\n",
    "使用numpy计算上述过程："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[19 22]\n",
      " [43 50]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "A = np.array([[1, 2],\n",
    "              [3, 4]])\n",
    "B = np.array([[5, 6],\n",
    "              [7, 8]])\n",
    "print(np.dot(A, B))\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.3神经网络的内积\n",
    "\n",
    "下面图中的神经网络省略了偏置项和激活函数，只有权重：\n",
    "\n",
    "![image-20230312092416623](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/image-20230312092416623.png)\n",
    "\n",
    "按照该图的神经网络，输入和输出满足如下公式：\n",
    "\n",
    "$$\n",
    "    \\begin{cases}\n",
    "    y_1=x_1w_1+x_2w_2\\\\\n",
    "    y_2=x_1w_3+x_2w_4\\\\\n",
    "    y_3=x_1w_5+x_2w_6\n",
    "    \\end{cases}\n",
    "$$\n",
    "\n",
    "运用线性代数知识，将该公式转为矩阵的乘法：\n",
    "\n",
    "$$\n",
    "    \\left[\n",
    "        \\begin{matrix}\n",
    "            w_1 &w_2\\\\\n",
    "            w_3 &w_4\\\\\n",
    "            w_5 &w_6\n",
    "        \\end{matrix}\n",
    "    \\right]\n",
    "    ·\n",
    "    \\left[\n",
    "        \\begin{matrix}\n",
    "            x_1\\\\\n",
    "               \\\\\n",
    "            x_2\n",
    "        \\end{matrix}\n",
    "    \\right]\n",
    "    =\n",
    "    \\left[\n",
    "        \\begin{matrix}\n",
    "            y_1\\\\\n",
    "            y_2\\\\\n",
    "            y_3\n",
    "        \\end{matrix}\n",
    "    \\right]\n",
    "    \\\\或\\\\\n",
    "    \\left[\n",
    "        \\begin{matrix}\n",
    "            x_1 &x_2\n",
    "        \\end{matrix}\n",
    "    \\right]    \n",
    "    ·\n",
    "    \\left[\n",
    "        \\begin{matrix}\n",
    "            w_1 &w_3 &w_5\\\\\n",
    "            w_2 &w_4 &w_6\n",
    "        \\end{matrix}\n",
    "    \\right]\n",
    "    =\n",
    "    \\left[\n",
    "        \\begin{matrix}\n",
    "            y_1 &y_2 &y_3\n",
    "        \\end{matrix}\n",
    "    \\right]    \n",
    "$$\n",
    "\n",
    "与图中对应就用第二种。\n",
    "\n",
    "即：神经网络的公式表示为：\n",
    "\n",
    "$$\n",
    "    X·W=Y\n",
    "$$\n",
    "\n",
    "这里面要注意的是，W的写法，作用在$x_1 x_2$上的权重是竖着写的，即W应该一列一列的看。"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.4三层神经网络实现\n",
    "\n",
    "以下图中的神经网络为例，实现从输入到输出的（前向）处理。\n",
    "\n",
    "![image-20230312100657785](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/image-20230312100657785.png)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.4.1相关符号\n",
    "\n",
    "![image-20230312101646036](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/image-20230312101646036.png)\n",
    "\n",
    "从上图中可以看到权重和隐藏层的神经元的右上角有一个“(1)”，它表示权重和神经元的层号（即第1层的权重、第1层的神经元）。此外，权重的右下角有两个数字，它们是后一层的神经元和前一层的神经元的索引号。如果考虑偏置项就如下图所示：\n",
    "\n",
    "![image-20230312104332979](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/image-20230312104332979.png)\n",
    "\n",
    "偏置的右下角的索引号只有一个。这是因为前一层的偏置神经元（神经元“1”）只有一个。\n",
    "用矩阵乘法运算表示第一层的加权和表示为：\n",
    "\n",
    "$$A^{(1)}=XW^{(1)}+B^{(1)}$$\n",
    "\n",
    "![image-20230312110805352](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/image-20230312110805352.png)\n",
    "\n",
    "下面用代码实现这个三层网络："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "X = np.array([1, 0.5])\n",
    "W1 = np.array([[0.1, 0.3, 0.5],\n",
    "               [0.2, 0.4, 0.6]])\n",
    "B1 = np.array([0.1, 0.2, 0.3])\n",
    "\n",
    "A1 = np.dot(X, W1)+B1\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "将上述的加权和，通过一个激活函数，示意图如下：\n",
    "\n",
    "![image-20230312125542372](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/image-20230312125542372.png)\n",
    "\n",
    "这里使用sigmoid激活函数："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.3 0.7 1.1]\n",
      "[0.57444252 0.66818777 0.75026011]\n"
     ]
    }
   ],
   "source": [
    "Z1 = sigmoid_function(A1)  # 第一层输出\n",
    "\n",
    "print(A1)\n",
    "print(Z1)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来编写第二层，第二层为3输入2输出，所以输入矩阵形状为1×3，输出矩阵为1×2，权重矩阵为3×2\n",
    "\n",
    "![image-20230312133625502](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/image-20230312133625502.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.51615984 1.21402696]\n",
      "[0.62624937 0.7710107 ]\n"
     ]
    }
   ],
   "source": [
    "W2 = np.array([[0.1, 0.4],\n",
    "               [0.2, 0.5],\n",
    "               [0.3, 0.6]])\n",
    "B2 = np.array([0.1, 0.2])\n",
    "A2 = np.dot(Z1, W2)+B2\n",
    "Z2 = sigmoid_function(A2)\n",
    "\n",
    "print(A2)\n",
    "print(Z2)\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "最后一层网络2输入2输出，输入/输出矩阵为1×2，权重矩阵为2×2，加权和直接输出。输出层的激活函数用$σ()$表示，不同于隐藏层的激活函数$h()$。\n",
    "\n",
    "![image-20230312133650502](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/image-20230312133650502.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.31682708 0.69627909]\n"
     ]
    }
   ],
   "source": [
    "W3 = np.array([[0.1, 0.3],\n",
    "               [0.2, 0.4]])\n",
    "B3 = np.array([0.1, 0.2])\n",
    "\n",
    "Y = np.dot(Z2, W3)+B3\n",
    "print(Y)\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.4.2代码小结\n",
    "\n",
    "把之前的所有步骤都总结一下，权重用大写字母表示，偏置项和中间结果都用小写字母表示。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.31682708 0.69627909]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "# 初始化网络参数\n",
    "def init_network():\n",
    "    network = {}\n",
    "    network['W1'] = np.array([[0.1, 0.3, 0.5],\n",
    "                              [0.2, 0.4, 0.6]])\n",
    "    network['b1'] = np.array([0.1, 0.2, 0.3])\n",
    "    network['W2'] = np.array([[0.1, 0.4],\n",
    "                              [0.2, 0.5],\n",
    "                              [0.3, 0.6]])\n",
    "    network['b2'] = np.array([0.1, 0.2])\n",
    "    network['W3'] = np.array([[0.1, 0.3],\n",
    "                              [0.2, 0.4]])\n",
    "    network['b3'] = np.array([0.1, 0.2])\n",
    "    return network\n",
    "\n",
    "# 前向传播\n",
    "def forward(network, x):\n",
    "    W1, W2, W3 = network['W1'], network['W2'], network['W3']\n",
    "    b1, b2, b3 = network['b1'], network['b2'], network['b3']\n",
    "\n",
    "    # 计算第一层\n",
    "    a1 = np.dot(x, W1)+b1\n",
    "    z1 = sigmoid_function(a1)\n",
    "\n",
    "    # 计算第二层\n",
    "    a2 = np.dot(z1, W2)+b2\n",
    "    z2 = sigmoid_function(a2)\n",
    "\n",
    "    # 计算第三层\n",
    "    a3 = np.dot(z2, W3)+b3\n",
    "    y = a3\n",
    "\n",
    "    return y\n",
    "\n",
    "\n",
    "# 执行网络运算\n",
    "network = init_network()\n",
    "x = np.array([1, 0.5])\n",
    "y = forward(network, x)\n",
    "print(y)\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "结果与之前一致。"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.5输出层的设计\n",
    "\n",
    "神经网络可以用在分类问题和回归问题上，不过需要根据情况改变输出层的激活函数。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.5.1恒等函数和softmax函数\n",
    "\n",
    "- 恒等函数会将输入按原样输出，对于输入的信息，不加以任何改动地直接输出。\n",
    "\n",
    "![image-20230312161434280](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/image-20230312161434280.png)\n",
    "\n",
    "- softmax函数可以用下面的式子表示\n",
    "\n",
    "![image-20230312161525102](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/image-20230312161525102.png)\n",
    "\n",
    "观察上式可以发现：其实就是对每个输入数据先取一次$exp$，然后将各自的结果求比例。\n",
    "\n",
    "![image-20230313095435611](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/image-20230313095435611.png)\n",
    "\n",
    "**问：既然最后是求比例，为什么不直接求比例，非要先取一次$exp$？**\n",
    "\n",
    "**答：因为这样可以使大的更大，小的更小，使输出结果两级分化，数据之间区别更明显，输出值大的最后得分更高，更有利于模型的学习。**\n",
    "\n",
    "**比如，输出1，2，3，4四个数，直接占比为0.1，0.2，0.3，0.4；但是经过softmax之后，输出变为0.03，0.08，0.23，0.64（近似）。**\n",
    "\n",
    "softmax函数实现："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.01821127 0.24519181 0.73659691]\n"
     ]
    }
   ],
   "source": [
    "def softmax_function(x):\n",
    "    tmp = np.exp(x)\n",
    "    denominator = np.sum(tmp)  # 求分母\n",
    "    y = tmp/denominator\n",
    "\n",
    "    return y\n",
    "\n",
    "\n",
    "# 测试\n",
    "print(softmax_function([0.3, 2.9, 4.0]))\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.5.2softmax函数的改进\n",
    "\n",
    "【问题】：如果一个网络中输出层输入数据$a_k=100$，会计算$e^{100}$，直接导致数据溢出。\n",
    "\n",
    "【解决】：所以在计算过程中要避免直接计算这种极大的数值，像如下方式改进：\n",
    "\n",
    "![image-20230313100841562](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/image-20230313100841562.png)\n",
    "\n",
    "也就是说，**在$exp$里面，上下任意加上同一个常数，最后的计算结果和原来一样**。这里的$C'$可以使用任何值，但是为了防止溢出，一般会使用**输入信号中的最大值**。\n",
    "\n",
    "代码编写如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[9.99954600e-01 4.53978686e-05 2.06106005e-09]\n"
     ]
    }
   ],
   "source": [
    "def softmax(a):\n",
    "    C = np.max(a)  # 先找到最大的那个\n",
    "    tmp = np.exp(a-C)\n",
    "    y = tmp/np.sum(tmp)\n",
    "\n",
    "    return y\n",
    "\n",
    "\n",
    "# 测试\n",
    "print(softmax([1010, 1000, 990]))\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.5.3softmax函数的特性\n",
    "\n",
    "经过softmax函数后的输出向量中，所有元素之和为1，例如：\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.01821127 0.24519181 0.73659691]\n",
      "1.0\n"
     ]
    }
   ],
   "source": [
    "a = np.array([0.3, 2.9, 4.0])\n",
    "y = softmax(a)\n",
    "\n",
    "print(y)\n",
    "print(np.sum(y))\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "正因为每个元素的大小都在0~1之间，所以也可以把输出的结果称为**概率**。可以解释成y[0]的概率是0.018（1.8%），y[1]的概率是0.245（24.5%），y[2]的概率是0.737（73.7%）。\n",
    "\n",
    "在分类问题中，按照上述的结果，y[2]的概率最大，所以可以判断该结果为第2类。也就是说，通过**使用softmax函数，我们可以用概率的（统计的）方法处理问题**。\n",
    "\n",
    "需要注意的是，即便使用了softmax函数，**各个元素之间的大小关系也不会改变**。比如，a的最大值是第2个元素，y的最大值也仍是第2个元素。\n",
    "\n",
    "一般而言，神经网络只把输出值最大的神经元所对应的类别作为识别结果。并且，即便使用softmax函数，输出值最大的神经元的位置也不会变。因此，神经网络在进行分类时，输出层的softmax函数可以省略。在实际的问题中，由于指数函数的运算需要一定的计算机运算量，因此输出层的softmax函数一般会被省略。\n",
    "\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.5.4输出层神经元的数量\n",
    "\n",
    "输出的神经元数量表示了分类的类别数。比如，对一张图像中的数字进行识别，数字一共有10类（0~9），所以最后输出层应该有10个神经元。\n",
    "\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.6手写数字识别\n",
    "\n",
    "前向传播（推理阶段）：即网络参数都已学习完毕，输入数据并从网络输入层到输出层计算出对应输出，对输出数据（概率）大小进行排序，得出预测结果。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.6.1MNIST数据集\n",
    "\n",
    "MNIST数据集是由0到9的数字图像构成的（图3-24）。训练图像有6万张，测试图像有1万张，这些图像可以用于学习和推理。MNIST数据集的一般使用方法是，先用训练图像进行学习，再用学习到的模型度量能在多大程度上对测试图像进行正确的分类。\n",
    "\n",
    "![image-20230313185401788](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/image-20230313185401788.png)\n",
    "\n",
    "- MNIST数据集的特点：\n",
    "\n",
    "    1. 28×28像素的灰度图\n",
    "\n",
    "    2. 每个图像数据都相应地标有“7”“2”“1”等标签。\n",
    "\n",
    "- 如何使用MNIST数据集？\n",
    "\n",
    "    1. 去图灵网（ http://www.ituring.com.cn/book/1921 ）下载本书配套的代码（点击右侧【随书下载字样】）。\n",
    "\n",
    "    2. 下载完后，解压刚刚下载的zip文件，解压完成后里面包含如下图的这些文件夹，把这些文件夹的上一级目录复制到你自己创建的文件夹中，并改成英文名。\n",
    "\n",
    "        ![image-20230409112026051](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/image-20230409112026051.png)\n",
    "\n",
    "        复制完成后如下图，我把复制完的文件夹改名成Demo，刚刚下载的配套代码全部放在Demo文件夹里：\n",
    "\n",
    "        ![image-20230409111945227](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/image-20230409111945227.png)\n",
    "\n",
    "        ![image-20230409111916350](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/image-20230409111916350.png)\n",
    "\n",
    "    3. 用编译器打开工作文件夹（就是你存放所有与该项目有关的文件的文件夹），我这里是DeepLearningNoob，里面包含刚刚下载的源码Demo文件夹，以及一些的记笔记用的jupyter notebook文件。\n",
    "\n",
    "        ![image-20230409112646588](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/image-20230409112646588.png)\n",
    "\n",
    "    4. 到这一步，基本的文件夹结构就已经弄好了，现在看一下如何在程序里使用Demo文件夹里刚刚下载的那些py文件。\n",
    "        \n",
    "        这就要解释一下python中的sys和os两个模块了。\n",
    "\n",
    "        从外部import模块的时候，首先要让python知道你要导入的模块在什么位置（即这些模块在哪个文件夹里），而这些“位置”是保存在sys.path这个系统变量里的（sys.path是一个list，因此也支持append操作）。打印sys.path看一下这个list里面包含什么内容：\n",
    "\n",
    "        ```py\n",
    "        import sys,os\n",
    "\n",
    "        print(sys.path)\n",
    "        ```\n",
    "\n",
    "        ![image-20230409114328516](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/image-20230409114328516.png)\n",
    "\n",
    "        在import某个模块的时候，python会到sys.path的路径中找导入的模块，找到了就直接使用。对于外部模块（比如Demo/dataset/mnist.py）可以看到路径是不在sys.path中的，所以此时就要把Demo路径添加到sys.path中。\n",
    "\n",
    "        如何获取Demo的路径呢？\n",
    "\n",
    "        如果是Demo文件夹里的py文件，就在文件开头添加如下：\n",
    "\n",
    "        ```py\n",
    "        import sys,os\n",
    "\n",
    "        current_file_dir = os.path.dirname(__file__)   # 获取当前文件所在目录\n",
    "        os.chdir(current_file_dir)  # 更改工作空间路径\n",
    "        sys.path.append(os.path.abspath(os.pardir))  # 添加路径\n",
    "        ```\n",
    "\n",
    "        如果是DeepLearningNoob文件夹下的ipynb文件中的代码块，就在代码块中添加如下：\n",
    "\n",
    "        ```py\n",
    "        import sys,os\n",
    "\n",
    "        current_file_path = os.path.dirname(os.path.realpath('__file__')) # 获取ipynb文件的路径\n",
    "        Demo_path = os.path.join(current_file_path, 'Demo') # 组合成Demo文件夹所在路径\n",
    "        sys.path.append(Demo_path) # 添加路径\n",
    "\n",
    "        ```"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.6.2MNIST数据集的使用\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "#### 3.6.2.1导出数据\n",
    "\n",
    "- load_mnist：以<b>（训练图像,训练标签），(测试图像,测试标签)</b>的形式返回读入的MNIST数据。\n",
    "\n",
    "    - normalize：是否将输入图像正规化为0.0～1.0的值。如果将该参数设置为False，则输入图像的像素会保持原来的0～255。\n",
    "\n",
    "    - one_hot_label：False时，只是像7、2这样简单保存正确解标签；True时，标签则保存为one-hot表示。one-hot数组是指[0,0,1,0,0,0,0,0,0,0]这样的数组。\n",
    "\n",
    "    - flatten：是否将图像展开为一维数组。即：如果将该参数设置为False，则输入图像为1 × 28 × 28的三维数组；若设置为True，则输入图像会保存为由784个元素构成的一维数组。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(60000, 784)\n",
      "(60000,)\n",
      "(10000, 784)\n",
      "(10000,)\n"
     ]
    }
   ],
   "source": [
    "import sys,os\n",
    "\n",
    "current_file_path = os.path.dirname(os.path.realpath('__file__')) # .\\DeepLearningNoob\\\n",
    "\n",
    "Demo_path = os.path.join(current_file_path, 'Demo') # .\\DeepLearningNoob\\Demo\n",
    "sys.path.append(Demo_path)\n",
    "\n",
    "from dataset.mnist import load_mnist\n",
    "\n",
    "# 导出数据\n",
    "(x_train, y_train), (x_test, y_test) = load_mnist(flatten=True, normalize=False)\n",
    "\n",
    "# 查看数据的shape\n",
    "print(x_train.shape)  # 因为flatten是true，所以训练图像是60000张28×28=784的图像。\n",
    "print(y_train.shape)  # 60000张图片对应60000个标签\n",
    "print(x_test.shape)  # 10000张测试图像\n",
    "print(y_test.shape)  # 10000张测试标签\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.6.2.2显示图像\n",
    "\n",
    "需要用到PIL（Python Image Library）\n",
    "\n",
    "- Image.fromarray(np.uint8(img))\n",
    "\n",
    "    - 从数据集读取到的图像数据是保存在array数组里的，但是这种形式的数据不能直接使用，需要转为image对象数据。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5\n",
      "(784,)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "from PIL import Image\n",
    "\n",
    "img = x_train[0]  # 训练图像的第一张\n",
    "label = y_train[0]  # 对应标签\n",
    "print(label)\n",
    "print(img.shape)\n",
    "\n",
    "img = img.reshape(28, 28)\n",
    "pil_img = Image.fromarray(np.uint8(img))  # array转image\n",
    "pil_img.show()\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.6.2.3神经网络的前向传播\n",
    "\n",
    "因为是对一张图片的结果进行识别，输入的对象是一张图片，在MNIST数据集里，其像素为28×28，像素总数为784，所以**输入的神经元个数为784**；要判断是哪个数字，所以类别分别是0~9这10个数字，所以<b>输出神经元个数为10</b>。这样，输入层和输出层的结构就确定好了。\n",
    "\n",
    "接下来是隐藏层（中间层）。设置两个隐藏层，神经元个数分别是50和100。（50、100可以设置为其他任意值）。\n",
    "\n",
    "- get_data()：在数据集中获取数据，返回**测试图像，测试标签**。\n",
    "\n",
    "- init_network()：读入保存在pickle文件中的权重参数（pickle可以将程序运行中的对象保存为文件）。这个文件中以字典变量的形式保存了权重和偏置参数。\n",
    "\n",
    "- predict(network,x)：搭建网络结构，并对输入数据进行计算，得出输出结果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pickle\n",
    "\n",
    "def get_data():\n",
    "    (x_train, y_train), (x_test, y_test) = load_mnist(\n",
    "        normalize=True, flatten=True, one_hot_label=False)\n",
    "    return x_test, y_test\n",
    "\n",
    "def init_network():\n",
    "    # 权重文件sample_weight.pkl在ch03里\n",
    "    with open('./Demo/ch03/sample_weight.pkl', 'rb') as f:\n",
    "        network = pickle.load(f)\n",
    "        return network\n",
    "\n",
    "\n",
    "def predict(network, x):\n",
    "    # 获取权重即偏置项\n",
    "    W1, W2, W3 = network['W1'], network['W2'], network['W3']\n",
    "    b1, b2, b3 = network['b1'], network['b2'], network['b3']\n",
    "    # 搭建网络\n",
    "    a1 = np.dot(x, W1)+b1\n",
    "    z1 = sigmoid_function(a1)\n",
    "    a2 = np.dot(z1, W2)+b2\n",
    "    z2 = sigmoid_function(a2)\n",
    "    a3 = np.dot(z2, W3)+b3\n",
    "    y = softmax(a3)\n",
    "\n",
    "    return y\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "【题外话】：`with open as f`的用法\n",
    "\n",
    "由于文件读写时都有可能产生`IOError`，一旦出错，后面的`f.close()`就不会调用。所以，为了保证无论是否出错都能正确地关闭文件，我们可以使用`try ... finally`来实现：\n",
    "\n",
    "```py\n",
    "try:\n",
    "    f = open('/path/to/file', 'r')\n",
    "    print(f.read())\n",
    "finally:\n",
    "    if f:\n",
    "        f.close()\n",
    "```\n",
    "\n",
    "但是每次都这么写实在太繁琐，所以，Python引入了`with`语句来自动帮我们调用`close()`方法：\n",
    "\n",
    "```py\n",
    "with open('/path/to/file', 'r') as f:\n",
    "    print(f.read())\n",
    "```\n",
    "\n",
    "这和前面的`try ... finally`是一样的，但是代码更佳简洁，并且不必调用`f.close()`方法。\n",
    "\n",
    "对测试集用上面的网络进行预测："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(10000, 784)\n",
      "[7 2 1 ... 4 5 6]\n",
      "[7. 2. 1. ... 4. 5. 6.]\n",
      "0.9352\n"
     ]
    }
   ],
   "source": [
    "x_test, y_test = get_data()  # 获取数据\n",
    "network = init_network()  # 获取网络参数\n",
    "\n",
    "x = np.array(x_test)  # 输入图像\n",
    "label = np.array(y_test)  # 标签\n",
    "predict_result = np.zeros(label.shape)  # 存放预测结果\n",
    "\n",
    "# 神经网络计算\n",
    "for i in range(len(x)):\n",
    "    y = predict(network, x[i])\n",
    "    p = np.argmax(y)  # 获取y中最大值对应的索引\n",
    "    predict_result[i] = p\n",
    "\n",
    "print(label)\n",
    "print(len(predict_result))\n",
    "print(np.sum(label == predict_result)/len(label))\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.6.2.4 批处理\n",
    "\n",
    "为了加快处理速度，可以一次性送入若干张图片进行计算，而不需要像上面那样一次送一张。\n",
    "\n",
    "因为大多数处理数值计算的库都进行了能够高效处理大型数组运算的优化。\n",
    "\n",
    "基于之前的代码，对其稍作修改："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.9352\n"
     ]
    }
   ],
   "source": [
    "x_test, y_test = get_data()  # 获取数据\n",
    "network = init_network()  # 获取网络参数\n",
    "\n",
    "batch_size = 100  # 一次送100个数据进入网络\n",
    "accuraccy_cnt = 0\n",
    "\n",
    "for i in range(0, len(x_test), batch_size):\n",
    "    y_batch = predict(network, x_test[i:i+batch_size])\n",
    "    p = np.argmax(y_batch, axis=1)\n",
    "    accuraccy_cnt = accuraccy_cnt+np.sum(p == y_test[i:i+batch_size])\n",
    "\n",
    "print(accuraccy_cnt/len(x_test))\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "【题外话】：argmax()中的axis参数\n",
    "\n",
    "![二维矩阵](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/2020081016403863.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1 2 1 0]\n",
      "[3 0 1]\n"
     ]
    }
   ],
   "source": [
    "x = np.array([[0.1, 0.8, 0.1],\n",
    "              [0.3, 0.1, 0.6],\n",
    "              [0.2, 0.5, 0.3],\n",
    "              [0.8, 0.1, 0.1]])\n",
    "print(np.argmax(x, axis=1))  # 沿着1轴的方向看（横着看）\n",
    "print(np.argmax(x, axis=0))  # 沿着0轴的方向看（竖着看）\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "pythonProject",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
